Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.910461
Epoch 1
Validation binary_cross_entropy = 89.341003
Epoch 2
Validation binary_cross_entropy = 3.847185
Epoch 3
Validation binary_cross_entropy = 6.529900
Epoch 4
Loss = 6.6467e+01, PNorm = 60.3403, GNorm = 1281.2727, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 27.990278
Epoch 5
Validation binary_cross_entropy = 8.324463
Epoch 6
Validation binary_cross_entropy = 8.248922
Epoch 7
Validation binary_cross_entropy = 14.407098
Epoch 8
Validation binary_cross_entropy = 12.128242
Epoch 9
Loss = 7.7263e+01, PNorm = 60.4702, GNorm = 1792.8138, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 8.890367
Epoch 10
Validation binary_cross_entropy = 5.956184
Epoch 11
Validation binary_cross_entropy = 16.128309
Epoch 12
Validation binary_cross_entropy = 13.710227
Epoch 13
Validation binary_cross_entropy = 6.214542
Epoch 14
Loss = 5.6713e+01, PNorm = 60.6370, GNorm = 995.8414, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 5.911929
Epoch 15
Validation binary_cross_entropy = 18.795019
Epoch 16
Validation binary_cross_entropy = 18.087387
Epoch 17
Validation binary_cross_entropy = 8.776024
Epoch 18
Validation binary_cross_entropy = 8.232661
Epoch 19
Loss = 4.0212e+01, PNorm = 60.7741, GNorm = 755.9925, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 12.843075
Epoch 20
Validation binary_cross_entropy = 20.478848
Epoch 21
Validation binary_cross_entropy = 6.206604
Epoch 22
Validation binary_cross_entropy = 5.541602
Epoch 23
Validation binary_cross_entropy = 33.413334
Epoch 24
Loss = 5.1353e+01, PNorm = 60.9114, GNorm = 1232.3980, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 56.602737
Epoch 25
Validation binary_cross_entropy = 18.482014
Epoch 26
Validation binary_cross_entropy = 5.500941
Epoch 27
Validation binary_cross_entropy = 5.712425
Epoch 28
Validation binary_cross_entropy = 13.614973
Epoch 29
Loss = 5.2966e+01, PNorm = 61.0150, GNorm = 966.7051, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 25.822315
Epoch 30
Validation binary_cross_entropy = 14.657166
Epoch 31
Validation binary_cross_entropy = 4.417408
Epoch 32
Validation binary_cross_entropy = 5.219846
Epoch 33
Validation binary_cross_entropy = 9.286506
Epoch 34
Loss = 2.6459e+01, PNorm = 61.1137, GNorm = 684.2135, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 10.716827
Epoch 35
Validation binary_cross_entropy = 8.794614
Epoch 36
Validation binary_cross_entropy = 5.557949
Epoch 37
Validation binary_cross_entropy = 4.835972
Epoch 38
Validation binary_cross_entropy = 4.795530
Epoch 39
Loss = 3.1661e+01, PNorm = 61.2061, GNorm = 918.9182, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 3.855653
Epoch 40
Validation binary_cross_entropy = 3.689294
Epoch 41
Validation binary_cross_entropy = 6.823014
Epoch 42
Validation binary_cross_entropy = 13.548906
Epoch 43
Validation binary_cross_entropy = 4.529182
Epoch 44
Loss = 2.7725e+01, PNorm = 61.2784, GNorm = 640.4585, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 3.668518
Epoch 45
Validation binary_cross_entropy = 8.159622
Epoch 46
Validation binary_cross_entropy = 9.279360
Epoch 47
Validation binary_cross_entropy = 4.472158
Epoch 48
Validation binary_cross_entropy = 3.593637
Epoch 49
Loss = 1.6572e+01, PNorm = 61.3409, GNorm = 595.1940, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 4.504452
Epoch 50
Validation binary_cross_entropy = 12.896179
Epoch 51
Validation binary_cross_entropy = 22.457830
Epoch 52
Validation binary_cross_entropy = 11.010756
Epoch 53
Validation binary_cross_entropy = 8.186440
Epoch 54
Loss = 1.3899e+01, PNorm = 61.3940, GNorm = 378.3897, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 3.110444
Epoch 55
Validation binary_cross_entropy = 3.165883
Epoch 56
Validation binary_cross_entropy = 4.724512
Epoch 57
Validation binary_cross_entropy = 7.716751
Epoch 58
Validation binary_cross_entropy = 5.449252
Epoch 59
Loss = 1.2182e+01, PNorm = 61.4467, GNorm = 320.5055, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 3.657795
Epoch 60
Validation binary_cross_entropy = 3.214944
Epoch 61
Validation binary_cross_entropy = 3.611051
Epoch 62
Validation binary_cross_entropy = 6.009264
Epoch 63
Validation binary_cross_entropy = 6.124551
Epoch 64
Loss = 9.8409e+00, PNorm = 61.4948, GNorm = 285.8431, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 7.487165
Epoch 65
Validation binary_cross_entropy = 6.776883
Epoch 66
Validation binary_cross_entropy = 4.355065
Epoch 67
Validation binary_cross_entropy = 3.091598
Epoch 68
Validation binary_cross_entropy = 3.068873
Epoch 69
Loss = 1.1951e+01, PNorm = 61.5289, GNorm = 365.1926, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 6.338405
Epoch 70
Validation binary_cross_entropy = 9.582033
Epoch 71
Validation binary_cross_entropy = 5.595101
Epoch 72
Validation binary_cross_entropy = 4.649297
Epoch 73
Validation binary_cross_entropy = 4.618534
Epoch 74
Loss = 1.2613e+01, PNorm = 61.5436, GNorm = 494.1050, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.065441
Epoch 75
Validation binary_cross_entropy = 4.589284
Epoch 76
Validation binary_cross_entropy = 6.032799
Epoch 77
Validation binary_cross_entropy = 9.060344
Epoch 78
Validation binary_cross_entropy = 12.102522
Epoch 79
Loss = 1.4177e+01, PNorm = 61.5801, GNorm = 298.8629, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 10.379003
Epoch 80
Validation binary_cross_entropy = 4.738612
Epoch 81
Validation binary_cross_entropy = 2.803349
Epoch 82
Validation binary_cross_entropy = 3.951131
Epoch 83
Validation binary_cross_entropy = 7.581089
Epoch 84
Loss = 8.5500e+00, PNorm = 61.6055, GNorm = 271.8676, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 10.395691
Epoch 85
Validation binary_cross_entropy = 10.289286
Epoch 86
Validation binary_cross_entropy = 6.294043
Epoch 87
Validation binary_cross_entropy = 3.100497
Epoch 88
Validation binary_cross_entropy = 3.035665
Epoch 89
Loss = 3.7892e+00, PNorm = 61.6392, GNorm = 169.5024, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 2.971715
Epoch 90
Validation binary_cross_entropy = 2.925759
Epoch 91
Validation binary_cross_entropy = 2.961452
Epoch 92
Validation binary_cross_entropy = 3.081686
Epoch 93
Validation binary_cross_entropy = 3.206463
Epoch 94
Loss = 6.4887e+00, PNorm = 61.6787, GNorm = 355.1301, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 3.007054
Epoch 95
Validation binary_cross_entropy = 1.790497
Epoch 96
Validation binary_cross_entropy = 1.555956
Epoch 97
Validation binary_cross_entropy = 1.412401
Epoch 98
Validation binary_cross_entropy = 1.515740
Epoch 99
Loss = 5.6776e+00, PNorm = 61.7180, GNorm = 205.0289, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.533211
Epoch 100
Validation binary_cross_entropy = 2.460125
Epoch 101
Validation binary_cross_entropy = 2.848896
Epoch 102
Validation binary_cross_entropy = 2.463755
Epoch 103
Validation binary_cross_entropy = 1.439328
Epoch 104
Loss = 6.8999e+00, PNorm = 61.7766, GNorm = 244.2709, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.667283
Epoch 105
Validation binary_cross_entropy = 0.673856
Epoch 106
Validation binary_cross_entropy = 0.729735
Epoch 107
Validation binary_cross_entropy = 0.730133
Epoch 108
Validation binary_cross_entropy = 1.147681
Epoch 109
Loss = 6.5165e+00, PNorm = 61.7840, GNorm = 224.8153, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 4.265874
Epoch 110
Validation binary_cross_entropy = 6.868631
Epoch 111
Validation binary_cross_entropy = 7.869038
Epoch 112
Validation binary_cross_entropy = 6.441723
Epoch 113
Validation binary_cross_entropy = 3.733810
Epoch 114
Loss = 6.6883e+00, PNorm = 61.7987, GNorm = 218.3858, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.633852
Epoch 115
Validation binary_cross_entropy = 0.821734
Epoch 116
Validation binary_cross_entropy = 1.605399
Epoch 117
Validation binary_cross_entropy = 2.543072
Epoch 118
Validation binary_cross_entropy = 3.947809
Epoch 119
Loss = 2.4223e+00, PNorm = 61.8330, GNorm = 134.9952, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 4.602258
Epoch 120
Validation binary_cross_entropy = 4.539103
Epoch 121
Validation binary_cross_entropy = 3.755602
Epoch 122
Validation binary_cross_entropy = 2.328201
Epoch 123
Validation binary_cross_entropy = 1.512186
Epoch 124
Loss = 1.0010e+01, PNorm = 61.8966, GNorm = 267.8123, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.651942
Epoch 125
Validation binary_cross_entropy = 0.554430
Epoch 126
Validation binary_cross_entropy = 0.535473
Epoch 127
Validation binary_cross_entropy = 0.607712
Epoch 128
Validation binary_cross_entropy = 0.836330
Epoch 129
Loss = 4.4010e+00, PNorm = 61.9469, GNorm = 221.5227, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.354166
Epoch 130
Validation binary_cross_entropy = 1.850498
Epoch 131
Validation binary_cross_entropy = 1.638687
Epoch 132
Validation binary_cross_entropy = 1.175876
Epoch 133
Validation binary_cross_entropy = 0.564962
Epoch 134
Loss = 4.1555e+00, PNorm = 61.9632, GNorm = 295.9409, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.449344
Epoch 135
Validation binary_cross_entropy = 0.482933
Epoch 136
Validation binary_cross_entropy = 0.513945
Epoch 137
Validation binary_cross_entropy = 0.547888
Epoch 138
Validation binary_cross_entropy = 0.554916
Epoch 139
Loss = 3.4655e+00, PNorm = 61.9583, GNorm = 131.1165, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.571913
Epoch 140
Validation binary_cross_entropy = 0.633584
Epoch 141
Validation binary_cross_entropy = 0.832653
Epoch 142
Validation binary_cross_entropy = 0.751984
Epoch 143
Validation binary_cross_entropy = 0.572729
Epoch 144
Loss = 6.6103e+00, PNorm = 61.9689, GNorm = 112.3096, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.526079
Epoch 145
Validation binary_cross_entropy = 0.504761
Epoch 146
Validation binary_cross_entropy = 0.518665
Epoch 147
Validation binary_cross_entropy = 0.524484
Epoch 148
Validation binary_cross_entropy = 0.513750
Epoch 149
Loss = 3.0270e+00, PNorm = 61.9930, GNorm = 134.2641, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.495053
Epoch 150
Validation binary_cross_entropy = 0.497362
Epoch 151
Validation binary_cross_entropy = 0.521780
Epoch 152
Validation binary_cross_entropy = 0.493672
Epoch 153
Validation binary_cross_entropy = 0.449434
Epoch 154
Loss = 3.1551e+00, PNorm = 62.0239, GNorm = 118.8248, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.435971
Epoch 155
Validation binary_cross_entropy = 0.433308
Epoch 156
Validation binary_cross_entropy = 0.453926
Epoch 157
Validation binary_cross_entropy = 0.482973
Epoch 158
Validation binary_cross_entropy = 0.519613
Epoch 159
Loss = 3.1828e+00, PNorm = 62.0398, GNorm = 156.2202, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.504899
Epoch 160
Validation binary_cross_entropy = 0.484482
Epoch 161
Validation binary_cross_entropy = 0.428327
Epoch 162
Validation binary_cross_entropy = 0.402460
Epoch 163
Validation binary_cross_entropy = 0.388448
Epoch 164
Loss = 3.0253e+00, PNorm = 62.0726, GNorm = 188.3140, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.385367
Epoch 165
Validation binary_cross_entropy = 0.398464
Epoch 166
Validation binary_cross_entropy = 0.457042
Epoch 167
Validation binary_cross_entropy = 0.623575
Epoch 168
Validation binary_cross_entropy = 0.788521
Epoch 169
Loss = 3.1802e+00, PNorm = 62.0826, GNorm = 120.0925, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.703143
Epoch 170
Validation binary_cross_entropy = 0.593909
Epoch 171
Validation binary_cross_entropy = 0.491079
Epoch 172
Validation binary_cross_entropy = 0.419035
Epoch 173
Validation binary_cross_entropy = 0.393560
Epoch 174
Loss = 3.3543e+00, PNorm = 62.0851, GNorm = 104.0858, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.392282
Epoch 175
Validation binary_cross_entropy = 0.434162
Epoch 176
Validation binary_cross_entropy = 0.507300
Epoch 177
Validation binary_cross_entropy = 0.588317
Epoch 178
Validation binary_cross_entropy = 0.670484
Epoch 179
Loss = 3.0236e+00, PNorm = 62.0904, GNorm = 149.1520, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.707306
Epoch 180
Validation binary_cross_entropy = 0.665461
Epoch 181
Validation binary_cross_entropy = 0.574781
Epoch 182
Validation binary_cross_entropy = 0.448036
Epoch 183
Validation binary_cross_entropy = 0.385206
Epoch 184
Loss = 2.5767e+00, PNorm = 62.1144, GNorm = 122.7632, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.371911
Epoch 185
Validation binary_cross_entropy = 0.380030
Epoch 186
Validation binary_cross_entropy = 0.395403
Epoch 187
Validation binary_cross_entropy = 0.428684
Epoch 188
Validation binary_cross_entropy = 0.483550
Epoch 189
Loss = 1.5169e+00, PNorm = 62.1220, GNorm = 79.2810, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.559788
Epoch 190
Validation binary_cross_entropy = 0.634187
Epoch 191
Validation binary_cross_entropy = 0.750728
Epoch 192
Validation binary_cross_entropy = 0.868408
Epoch 193
Validation binary_cross_entropy = 0.874858
Epoch 194
Loss = 3.3948e+00, PNorm = 62.1509, GNorm = 105.8627, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.797507
Epoch 195
Validation binary_cross_entropy = 0.685005
Epoch 196
Validation binary_cross_entropy = 0.625801
Epoch 197
Validation binary_cross_entropy = 0.558389
Epoch 198
Validation binary_cross_entropy = 0.539303
Epoch 199
Loss = 1.4831e+00, PNorm = 62.1937, GNorm = 114.2758, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.567428
Epoch 200
Validation binary_cross_entropy = 0.676321
Epoch 201
Validation binary_cross_entropy = 0.781197
Epoch 202
Validation binary_cross_entropy = 0.948087
Epoch 203
Validation binary_cross_entropy = 1.054684
Epoch 204
Loss = 1.8388e+00, PNorm = 62.2045, GNorm = 74.0612, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.982248
Epoch 205
Validation binary_cross_entropy = 0.883941
Epoch 206
Validation binary_cross_entropy = 0.691075
Epoch 207
Validation binary_cross_entropy = 0.504387
Epoch 208
Validation binary_cross_entropy = 0.418862
Epoch 209
Loss = 1.4039e+00, PNorm = 62.2060, GNorm = 103.7964, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.389865
Epoch 210
Validation binary_cross_entropy = 0.395870
Epoch 211
Validation binary_cross_entropy = 0.427356
Epoch 212
Validation binary_cross_entropy = 0.511054
Epoch 213
Validation binary_cross_entropy = 0.585323
Epoch 214
Loss = 1.7830e+00, PNorm = 62.1998, GNorm = 98.5987, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.653180
Epoch 215
Validation binary_cross_entropy = 0.716896
Epoch 216
Validation binary_cross_entropy = 0.747543
Epoch 217
Validation binary_cross_entropy = 0.725307
Epoch 218
Validation binary_cross_entropy = 0.699908
Epoch 219
Loss = 1.2918e+00, PNorm = 62.1975, GNorm = 83.1693, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.628302
Epoch 220
Validation binary_cross_entropy = 0.571904
Epoch 221
Validation binary_cross_entropy = 0.529801
Epoch 222
Validation binary_cross_entropy = 0.506011
Epoch 223
Validation binary_cross_entropy = 0.497977
Epoch 224
Loss = 2.2142e+00, PNorm = 62.2074, GNorm = 132.6817, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.483938
Epoch 225
Validation binary_cross_entropy = 0.466553
Epoch 226
Validation binary_cross_entropy = 0.462602
Epoch 227
Validation binary_cross_entropy = 0.479055
Epoch 228
Validation binary_cross_entropy = 0.503322
Epoch 229
Loss = 1.5228e+00, PNorm = 62.2242, GNorm = 67.9051, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.524068
Epoch 230
Validation binary_cross_entropy = 0.535169
Epoch 231
Validation binary_cross_entropy = 0.537549
Epoch 232
Validation binary_cross_entropy = 0.532834
Epoch 233
Validation binary_cross_entropy = 0.534241
Epoch 234
Loss = 1.0146e+00, PNorm = 62.2223, GNorm = 58.7325, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.537133
Epoch 235
Validation binary_cross_entropy = 0.545378
Epoch 236
Validation binary_cross_entropy = 0.553998
Epoch 237
Validation binary_cross_entropy = 0.556977
Epoch 238
Validation binary_cross_entropy = 0.566598
Epoch 239
Loss = 1.0428e+00, PNorm = 62.2282, GNorm = 50.6345, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.573437
Epoch 240
Validation binary_cross_entropy = 0.558531
Epoch 241
Validation binary_cross_entropy = 0.533822
Epoch 242
Validation binary_cross_entropy = 0.515724
Epoch 243
Validation binary_cross_entropy = 0.490006
Epoch 244
Loss = 1.2394e+00, PNorm = 62.2453, GNorm = 52.8484, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.476885
Epoch 245
Validation binary_cross_entropy = 0.465952
Epoch 246
Validation binary_cross_entropy = 0.475361
Epoch 247
Validation binary_cross_entropy = 0.488664
Epoch 248
Validation binary_cross_entropy = 0.494214
Epoch 249
Loss = 1.3997e+00, PNorm = 62.2674, GNorm = 84.6817, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.514652
Epoch 250
Validation binary_cross_entropy = 0.544662
Epoch 251
Validation binary_cross_entropy = 0.566423
Epoch 252
Validation binary_cross_entropy = 0.560779
Epoch 253
Validation binary_cross_entropy = 0.570117
Epoch 254
Loss = 1.1348e+00, PNorm = 62.2672, GNorm = 60.3477, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.568590
Epoch 255
Validation binary_cross_entropy = 0.547966
Epoch 256
Validation binary_cross_entropy = 0.533175
Epoch 257
Validation binary_cross_entropy = 0.490946
Epoch 258
Validation binary_cross_entropy = 0.470964
Epoch 259
Loss = 1.5534e+00, PNorm = 62.2871, GNorm = 91.5006, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.471613
Epoch 260
Validation binary_cross_entropy = 0.493444
Epoch 261
Validation binary_cross_entropy = 0.535710
Epoch 262
Validation binary_cross_entropy = 0.563791
Epoch 263
Validation binary_cross_entropy = 0.585239
Epoch 264
Loss = 1.1831e+00, PNorm = 62.2930, GNorm = 84.7628, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.579724
Epoch 265
Validation binary_cross_entropy = 0.531380
Epoch 266
Validation binary_cross_entropy = 0.485713
Epoch 267
Validation binary_cross_entropy = 0.471197
Epoch 268
Validation binary_cross_entropy = 0.474755
Epoch 269
Loss = 1.1571e+00, PNorm = 62.3016, GNorm = 70.9447, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.499255
Epoch 270
Validation binary_cross_entropy = 0.549161
Epoch 271
Validation binary_cross_entropy = 0.590397
Epoch 272
Validation binary_cross_entropy = 0.617133
Epoch 273
Validation binary_cross_entropy = 0.619395
Epoch 274
Loss = 6.6139e-01, PNorm = 62.3010, GNorm = 41.9429, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.626520
Epoch 275
Validation binary_cross_entropy = 0.630811
Epoch 276
Validation binary_cross_entropy = 0.621389
Epoch 277
Validation binary_cross_entropy = 0.605052
Epoch 278
Validation binary_cross_entropy = 0.564307
Epoch 279
Loss = 1.4262e+00, PNorm = 62.3033, GNorm = 40.2641, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.518884
Epoch 280
Validation binary_cross_entropy = 0.485537
Epoch 281
Validation binary_cross_entropy = 0.467179
Epoch 282
Validation binary_cross_entropy = 0.457832
Epoch 283
Validation binary_cross_entropy = 0.461832
Epoch 284
Loss = 8.4832e-01, PNorm = 62.3116, GNorm = 74.0698, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.470544
Epoch 285
Validation binary_cross_entropy = 0.477865
Epoch 286
Validation binary_cross_entropy = 0.491126
Epoch 287
Validation binary_cross_entropy = 0.517006
Epoch 288
Validation binary_cross_entropy = 0.534060
Epoch 289
Loss = 8.3813e-01, PNorm = 62.3247, GNorm = 47.4421, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.547296
Epoch 290
Validation binary_cross_entropy = 0.553208
Epoch 291
Validation binary_cross_entropy = 0.555601
Epoch 292
Validation binary_cross_entropy = 0.552482
Epoch 293
Validation binary_cross_entropy = 0.550622
Epoch 294
Loss = 1.5297e+00, PNorm = 62.3365, GNorm = 70.8515, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.540768
Epoch 295
Validation binary_cross_entropy = 0.536583
Epoch 296
Validation binary_cross_entropy = 0.533499
Epoch 297
Validation binary_cross_entropy = 0.532365
Epoch 298
Validation binary_cross_entropy = 0.532978
Epoch 299
Loss = 1.0993e+00, PNorm = 62.3429, GNorm = 110.7669, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.534384
Model 0 best validation binary_cross_entropy = 0.371911 on epoch 184
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.445539
Ensemble test binary_cross_entropy = 0.445539
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.303146
Epoch 1
Validation binary_cross_entropy = 87.065399
Epoch 2
Validation binary_cross_entropy = 11.272651
Epoch 3
Validation binary_cross_entropy = 6.126692
Epoch 4
Loss = 8.2213e+01, PNorm = 60.3456, GNorm = 1223.2552, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 12.107832
Epoch 5
Validation binary_cross_entropy = 11.215134
Epoch 6
Validation binary_cross_entropy = 7.962401
Epoch 7
Validation binary_cross_entropy = 13.266094
Epoch 8
Validation binary_cross_entropy = 16.918314
Epoch 9
Loss = 4.5733e+01, PNorm = 60.4820, GNorm = 668.2916, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 15.376219
Epoch 10
Validation binary_cross_entropy = 8.721134
Epoch 11
Validation binary_cross_entropy = 8.078317
Epoch 12
Validation binary_cross_entropy = 9.872864
Epoch 13
Validation binary_cross_entropy = 5.662525
Epoch 14
Loss = 4.4694e+01, PNorm = 60.6521, GNorm = 996.0574, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 12.240262
Epoch 15
Validation binary_cross_entropy = 12.960868
Epoch 16
Validation binary_cross_entropy = 6.184336
Epoch 17
Validation binary_cross_entropy = 5.121884
Epoch 18
Validation binary_cross_entropy = 16.560972
Epoch 19
Loss = 4.2234e+01, PNorm = 60.7932, GNorm = 907.2931, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 29.228920
Epoch 20
Validation binary_cross_entropy = 12.400211
Epoch 21
Validation binary_cross_entropy = 8.103108
Epoch 22
Validation binary_cross_entropy = 8.189325
Epoch 23
Validation binary_cross_entropy = 8.626789
Epoch 24
Loss = 5.2258e+01, PNorm = 60.9207, GNorm = 1010.7806, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 8.488929
Epoch 25
Validation binary_cross_entropy = 6.377424
Epoch 26
Validation binary_cross_entropy = 6.682878
Epoch 27
Validation binary_cross_entropy = 17.202599
Epoch 28
Validation binary_cross_entropy = 7.322231
Epoch 29
Loss = 2.7802e+01, PNorm = 61.0576, GNorm = 691.6568, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.586504
Epoch 30
Validation binary_cross_entropy = 4.258311
Epoch 31
Validation binary_cross_entropy = 17.421627
Epoch 32
Validation binary_cross_entropy = 5.477580
Epoch 33
Validation binary_cross_entropy = 5.242327
Epoch 34
Loss = 2.5386e+01, PNorm = 61.1582, GNorm = 663.7036, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.651583
Epoch 35
Validation binary_cross_entropy = 4.137651
Epoch 36
Validation binary_cross_entropy = 3.979825
Epoch 37
Validation binary_cross_entropy = 4.623708
Epoch 38
Validation binary_cross_entropy = 3.991704
Epoch 39
Loss = 3.5536e+01, PNorm = 61.2512, GNorm = 660.4237, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 5.396742
Epoch 40
Validation binary_cross_entropy = 8.783403
Epoch 41
Validation binary_cross_entropy = 6.822172
Epoch 42
Validation binary_cross_entropy = 3.616701
Epoch 43
Validation binary_cross_entropy = 4.001291
Epoch 44
Loss = 2.3904e+01, PNorm = 61.3359, GNorm = 531.5288, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 3.193424
Epoch 45
Validation binary_cross_entropy = 3.306324
Epoch 46
Validation binary_cross_entropy = 6.097272
Epoch 47
Validation binary_cross_entropy = 3.462186
Epoch 48
Validation binary_cross_entropy = 3.473108
Epoch 49
Loss = 1.6975e+01, PNorm = 61.4267, GNorm = 522.7149, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.747832
Epoch 50
Validation binary_cross_entropy = 3.018366
Epoch 51
Validation binary_cross_entropy = 3.480399
Epoch 52
Validation binary_cross_entropy = 3.354437
Epoch 53
Validation binary_cross_entropy = 6.922029
Epoch 54
Loss = 1.4209e+01, PNorm = 61.5126, GNorm = 494.6225, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 5.108291
Epoch 55
Validation binary_cross_entropy = 3.067129
Epoch 56
Validation binary_cross_entropy = 3.154743
Epoch 57
Validation binary_cross_entropy = 2.831263
Epoch 58
Validation binary_cross_entropy = 3.938399
Epoch 59
Loss = 1.1706e+01, PNorm = 61.5534, GNorm = 249.5662, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 13.780974
Epoch 60
Validation binary_cross_entropy = 14.039186
Epoch 61
Validation binary_cross_entropy = 4.426027
Epoch 62
Validation binary_cross_entropy = 2.249934
Epoch 63
Validation binary_cross_entropy = 2.226975
Epoch 64
Loss = 1.2732e+01, PNorm = 61.6021, GNorm = 299.9474, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.987772
Epoch 65
Validation binary_cross_entropy = 7.208192
Epoch 66
Validation binary_cross_entropy = 12.659389
Epoch 67
Validation binary_cross_entropy = 7.382079
Epoch 68
Validation binary_cross_entropy = 3.649607
Epoch 69
Loss = 1.2438e+01, PNorm = 61.6382, GNorm = 489.9693, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.533394
Epoch 70
Validation binary_cross_entropy = 3.628659
Epoch 71
Validation binary_cross_entropy = 5.133107
Epoch 72
Validation binary_cross_entropy = 5.917107
Epoch 73
Validation binary_cross_entropy = 4.582785
Epoch 74
Loss = 8.5285e+00, PNorm = 61.6733, GNorm = 436.5443, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.800111
Epoch 75
Validation binary_cross_entropy = 6.408177
Epoch 76
Validation binary_cross_entropy = 3.382234
Epoch 77
Validation binary_cross_entropy = 2.122383
Epoch 78
Validation binary_cross_entropy = 2.254491
Epoch 79
Loss = 4.9327e+00, PNorm = 61.7101, GNorm = 289.4343, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.636132
Epoch 80
Validation binary_cross_entropy = 3.848790
Epoch 81
Validation binary_cross_entropy = 4.224014
Epoch 82
Validation binary_cross_entropy = 4.550903
Epoch 83
Validation binary_cross_entropy = 5.787011
Epoch 84
Loss = 7.0318e+00, PNorm = 61.7625, GNorm = 298.4276, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 4.098574
Epoch 85
Validation binary_cross_entropy = 2.096697
Epoch 86
Validation binary_cross_entropy = 1.857877
Epoch 87
Validation binary_cross_entropy = 1.757657
Epoch 88
Validation binary_cross_entropy = 4.461347
Epoch 89
Loss = 7.7939e+00, PNorm = 61.8245, GNorm = 228.1945, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 6.506552
Epoch 90
Validation binary_cross_entropy = 4.245139
Epoch 91
Validation binary_cross_entropy = 1.692974
Epoch 92
Validation binary_cross_entropy = 1.532949
Epoch 93
Validation binary_cross_entropy = 1.547513
Epoch 94
Loss = 3.6524e+00, PNorm = 61.8928, GNorm = 249.1332, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 2.897031
Epoch 95
Validation binary_cross_entropy = 5.353084
Epoch 96
Validation binary_cross_entropy = 4.577930
Epoch 97
Validation binary_cross_entropy = 2.184110
Epoch 98
Validation binary_cross_entropy = 1.455158
Epoch 99
Loss = 7.1003e+00, PNorm = 61.9416, GNorm = 265.2748, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.460070
Epoch 100
Validation binary_cross_entropy = 1.575406
Epoch 101
Validation binary_cross_entropy = 3.824716
Epoch 102
Validation binary_cross_entropy = 6.881123
Epoch 103
Validation binary_cross_entropy = 5.664914
Epoch 104
Loss = 7.8784e+00, PNorm = 61.9604, GNorm = 196.5252, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 2.471875
Epoch 105
Validation binary_cross_entropy = 1.572292
Epoch 106
Validation binary_cross_entropy = 0.932510
Epoch 107
Validation binary_cross_entropy = 0.854727
Epoch 108
Validation binary_cross_entropy = 0.909663
Epoch 109
Loss = 5.8997e+00, PNorm = 61.9930, GNorm = 161.4618, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.238215
Epoch 110
Validation binary_cross_entropy = 1.405237
Epoch 111
Validation binary_cross_entropy = 1.793072
Epoch 112
Validation binary_cross_entropy = 1.455698
Epoch 113
Validation binary_cross_entropy = 0.888213
Epoch 114
Loss = 3.7956e+00, PNorm = 62.0360, GNorm = 166.6893, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.667786
Epoch 115
Validation binary_cross_entropy = 0.692085
Epoch 116
Validation binary_cross_entropy = 0.627918
Epoch 117
Validation binary_cross_entropy = 0.865524
Epoch 118
Validation binary_cross_entropy = 2.820270
Epoch 119
Loss = 4.8810e+00, PNorm = 62.0724, GNorm = 206.7057, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 8.595821
Epoch 120
Validation binary_cross_entropy = 9.028359
Epoch 121
Validation binary_cross_entropy = 2.146364
Epoch 122
Validation binary_cross_entropy = 0.664217
Epoch 123
Validation binary_cross_entropy = 0.456046
Epoch 124
Loss = 5.1376e+00, PNorm = 62.1448, GNorm = 221.0217, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.424321
Epoch 125
Validation binary_cross_entropy = 0.487963
Epoch 126
Validation binary_cross_entropy = 1.113006
Epoch 127
Validation binary_cross_entropy = 2.261934
Epoch 128
Validation binary_cross_entropy = 2.093935
Epoch 129
Loss = 7.8710e+00, PNorm = 62.1837, GNorm = 422.8108, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.744705
Epoch 130
Validation binary_cross_entropy = 0.473282
Epoch 131
Validation binary_cross_entropy = 0.462771
Epoch 132
Validation binary_cross_entropy = 0.467312
Epoch 133
Validation binary_cross_entropy = 0.470662
Epoch 134
Loss = 4.2330e+00, PNorm = 62.2375, GNorm = 188.4032, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.480017
Epoch 135
Validation binary_cross_entropy = 0.482022
Epoch 136
Validation binary_cross_entropy = 0.519399
Epoch 137
Validation binary_cross_entropy = 0.494172
Epoch 138
Validation binary_cross_entropy = 0.473911
Epoch 139
Loss = 2.9859e+00, PNorm = 62.2863, GNorm = 162.1681, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.448575
Epoch 140
Validation binary_cross_entropy = 0.435737
Epoch 141
Validation binary_cross_entropy = 0.428769
Epoch 142
Validation binary_cross_entropy = 0.429072
Epoch 143
Validation binary_cross_entropy = 0.421747
Epoch 144
Loss = 4.4877e+00, PNorm = 62.3139, GNorm = 215.4812, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.404402
Epoch 145
Validation binary_cross_entropy = 0.393330
Epoch 146
Validation binary_cross_entropy = 0.385905
Epoch 147
Validation binary_cross_entropy = 0.392392
Epoch 148
Validation binary_cross_entropy = 0.412447
Epoch 149
Loss = 3.4567e+00, PNorm = 62.3531, GNorm = 161.3822, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.436031
Epoch 150
Validation binary_cross_entropy = 0.453304
Epoch 151
Validation binary_cross_entropy = 0.455054
Epoch 152
Validation binary_cross_entropy = 0.478269
Epoch 153
Validation binary_cross_entropy = 0.512318
Epoch 154
Loss = 1.6894e+00, PNorm = 62.3811, GNorm = 80.7243, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.554934
Epoch 155
Validation binary_cross_entropy = 0.602537
Epoch 156
Validation binary_cross_entropy = 0.663911
Epoch 157
Validation binary_cross_entropy = 0.718674
Epoch 158
Validation binary_cross_entropy = 0.630068
Epoch 159
Loss = 2.6026e+00, PNorm = 62.4198, GNorm = 131.5075, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.585803
Epoch 160
Validation binary_cross_entropy = 0.647937
Epoch 161
Validation binary_cross_entropy = 0.646868
Epoch 162
Validation binary_cross_entropy = 0.540892
Epoch 163
Validation binary_cross_entropy = 0.457420
Epoch 164
Loss = 3.3801e+00, PNorm = 62.4789, GNorm = 110.2616, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.475057
Epoch 165
Validation binary_cross_entropy = 0.476393
Epoch 166
Validation binary_cross_entropy = 0.453280
Epoch 167
Validation binary_cross_entropy = 0.441104
Epoch 168
Validation binary_cross_entropy = 0.489203
Epoch 169
Loss = 1.8776e+00, PNorm = 62.5138, GNorm = 118.3469, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.519447
Epoch 170
Validation binary_cross_entropy = 0.531928
Epoch 171
Validation binary_cross_entropy = 0.477993
Epoch 172
Validation binary_cross_entropy = 0.440303
Epoch 173
Validation binary_cross_entropy = 0.443962
Epoch 174
Loss = 2.8896e+00, PNorm = 62.5434, GNorm = 101.5509, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.468452
Epoch 175
Validation binary_cross_entropy = 0.475272
Epoch 176
Validation binary_cross_entropy = 0.482423
Epoch 177
Validation binary_cross_entropy = 0.515127
Epoch 178
Validation binary_cross_entropy = 0.570763
Epoch 179
Loss = 3.2364e+00, PNorm = 62.5793, GNorm = 230.7247, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.666844
Epoch 180
Validation binary_cross_entropy = 0.545438
Epoch 181
Validation binary_cross_entropy = 0.404698
Epoch 182
Validation binary_cross_entropy = 0.356396
Epoch 183
Validation binary_cross_entropy = 0.341038
Epoch 184
Loss = 2.9673e+00, PNorm = 62.6264, GNorm = 92.4363, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.323848
Epoch 185
Validation binary_cross_entropy = 0.328627
Epoch 186
Validation binary_cross_entropy = 0.347899
Epoch 187
Validation binary_cross_entropy = 0.487261
Epoch 188
Validation binary_cross_entropy = 0.783384
Epoch 189
Loss = 1.9521e+00, PNorm = 62.6463, GNorm = 131.8721, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.057162
Epoch 190
Validation binary_cross_entropy = 1.083383
Epoch 191
Validation binary_cross_entropy = 0.771492
Epoch 192
Validation binary_cross_entropy = 0.440953
Epoch 193
Validation binary_cross_entropy = 0.302480
Epoch 194
Loss = 2.1601e+00, PNorm = 62.6611, GNorm = 72.9641, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.300251
Epoch 195
Validation binary_cross_entropy = 0.328433
Epoch 196
Validation binary_cross_entropy = 0.378117
Epoch 197
Validation binary_cross_entropy = 0.421297
Epoch 198
Validation binary_cross_entropy = 0.433858
Epoch 199
Loss = 1.0971e+00, PNorm = 62.6631, GNorm = 70.2402, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.429371
Epoch 200
Validation binary_cross_entropy = 0.431707
Epoch 201
Validation binary_cross_entropy = 0.425473
Epoch 202
Validation binary_cross_entropy = 0.405595
Epoch 203
Validation binary_cross_entropy = 0.387085
Epoch 204
Loss = 2.5465e+00, PNorm = 62.7111, GNorm = 96.1701, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.370066
Epoch 205
Validation binary_cross_entropy = 0.372976
Epoch 206
Validation binary_cross_entropy = 0.415804
Epoch 207
Validation binary_cross_entropy = 0.523134
Epoch 208
Validation binary_cross_entropy = 0.623078
Epoch 209
Loss = 1.2121e+00, PNorm = 62.7162, GNorm = 77.1421, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.652346
Epoch 210
Validation binary_cross_entropy = 0.628269
Epoch 211
Validation binary_cross_entropy = 0.590478
Epoch 212
Validation binary_cross_entropy = 0.556621
Epoch 213
Validation binary_cross_entropy = 0.517749
Epoch 214
Loss = 1.3727e+00, PNorm = 62.7248, GNorm = 69.3306, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.487092
Epoch 215
Validation binary_cross_entropy = 0.481397
Epoch 216
Validation binary_cross_entropy = 0.485721
Epoch 217
Validation binary_cross_entropy = 0.491909
Epoch 218
Validation binary_cross_entropy = 0.510946
Epoch 219
Loss = 1.8927e+00, PNorm = 62.7314, GNorm = 158.0956, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.531377
Epoch 220
Validation binary_cross_entropy = 0.544912
Epoch 221
Validation binary_cross_entropy = 0.544758
Epoch 222
Validation binary_cross_entropy = 0.532451
Epoch 223
Validation binary_cross_entropy = 0.509776
Epoch 224
Loss = 1.4729e+00, PNorm = 62.7497, GNorm = 78.3513, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.487268
Epoch 225
Validation binary_cross_entropy = 0.448060
Epoch 226
Validation binary_cross_entropy = 0.427843
Epoch 227
Validation binary_cross_entropy = 0.407727
Epoch 228
Validation binary_cross_entropy = 0.406587
Epoch 229
Loss = 1.2303e+00, PNorm = 62.7718, GNorm = 123.5504, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.418745
Epoch 230
Validation binary_cross_entropy = 0.448624
Epoch 231
Validation binary_cross_entropy = 0.478365
Epoch 232
Validation binary_cross_entropy = 0.510589
Epoch 233
Validation binary_cross_entropy = 0.537475
Epoch 234
Loss = 1.5774e+00, PNorm = 62.7876, GNorm = 88.5938, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.532057
Epoch 235
Validation binary_cross_entropy = 0.518311
Epoch 236
Validation binary_cross_entropy = 0.512087
Epoch 237
Validation binary_cross_entropy = 0.487214
Epoch 238
Validation binary_cross_entropy = 0.487579
Epoch 239
Loss = 1.1085e+00, PNorm = 62.8188, GNorm = 75.2757, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.479722
Epoch 240
Validation binary_cross_entropy = 0.464792
Epoch 241
Validation binary_cross_entropy = 0.452813
Epoch 242
Validation binary_cross_entropy = 0.438086
Epoch 243
Validation binary_cross_entropy = 0.435241
Epoch 244
Loss = 1.1824e+00, PNorm = 62.8415, GNorm = 75.0260, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.431812
Epoch 245
Validation binary_cross_entropy = 0.431621
Epoch 246
Validation binary_cross_entropy = 0.446629
Epoch 247
Validation binary_cross_entropy = 0.469423
Epoch 248
Validation binary_cross_entropy = 0.486042
Epoch 249
Loss = 5.6187e-01, PNorm = 62.8471, GNorm = 41.3731, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.496326
Epoch 250
Validation binary_cross_entropy = 0.484038
Epoch 251
Validation binary_cross_entropy = 0.460338
Epoch 252
Validation binary_cross_entropy = 0.417766
Epoch 253
Validation binary_cross_entropy = 0.390945
Epoch 254
Loss = 6.5718e-01, PNorm = 62.8654, GNorm = 60.9100, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.392734
Epoch 255
Validation binary_cross_entropy = 0.414253
Epoch 256
Validation binary_cross_entropy = 0.424811
Epoch 257
Validation binary_cross_entropy = 0.435155
Epoch 258
Validation binary_cross_entropy = 0.420171
Epoch 259
Loss = 1.2015e+00, PNorm = 62.8729, GNorm = 77.7895, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.396068
Epoch 260
Validation binary_cross_entropy = 0.382719
Epoch 261
Validation binary_cross_entropy = 0.383261
Epoch 262
Validation binary_cross_entropy = 0.382403
Epoch 263
Validation binary_cross_entropy = 0.393597
Epoch 264
Loss = 1.0590e+00, PNorm = 62.8935, GNorm = 45.9307, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.429721
Epoch 265
Validation binary_cross_entropy = 0.462130
Epoch 266
Validation binary_cross_entropy = 0.487498
Epoch 267
Validation binary_cross_entropy = 0.498573
Epoch 268
Validation binary_cross_entropy = 0.493342
Epoch 269
Loss = 6.3488e-01, PNorm = 62.9072, GNorm = 36.7931, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.474688
Epoch 270
Validation binary_cross_entropy = 0.451986
Epoch 271
Validation binary_cross_entropy = 0.425698
Epoch 272
Validation binary_cross_entropy = 0.398061
Epoch 273
Validation binary_cross_entropy = 0.386301
Epoch 274
Loss = 6.1411e-01, PNorm = 62.9407, GNorm = 33.7456, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.378204
Epoch 275
Validation binary_cross_entropy = 0.379729
Epoch 276
Validation binary_cross_entropy = 0.408026
Epoch 277
Validation binary_cross_entropy = 0.450099
Epoch 278
Validation binary_cross_entropy = 0.484976
Epoch 279
Loss = 5.6742e-01, PNorm = 62.9510, GNorm = 37.8196, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.506903
Epoch 280
Validation binary_cross_entropy = 0.512209
Epoch 281
Validation binary_cross_entropy = 0.499992
Epoch 282
Validation binary_cross_entropy = 0.488318
Epoch 283
Validation binary_cross_entropy = 0.464722
Epoch 284
Loss = 8.8230e-01, PNorm = 62.9861, GNorm = 65.6914, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.434332
Epoch 285
Validation binary_cross_entropy = 0.414113
Epoch 286
Validation binary_cross_entropy = 0.408269
Epoch 287
Validation binary_cross_entropy = 0.438143
Epoch 288
Validation binary_cross_entropy = 0.472097
Epoch 289
Loss = 7.2436e-01, PNorm = 63.0081, GNorm = 61.1698, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.498471
Epoch 290
Validation binary_cross_entropy = 0.498817
Epoch 291
Validation binary_cross_entropy = 0.500562
Epoch 292
Validation binary_cross_entropy = 0.505011
Epoch 293
Validation binary_cross_entropy = 0.498579
Epoch 294
Loss = 8.6198e-01, PNorm = 63.0006, GNorm = 87.1385, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.473244
Epoch 295
Validation binary_cross_entropy = 0.443435
Epoch 296
Validation binary_cross_entropy = 0.423917
Epoch 297
Validation binary_cross_entropy = 0.417529
Epoch 298
Validation binary_cross_entropy = 0.422747
Epoch 299
Loss = 7.6713e-01, PNorm = 63.0093, GNorm = 44.3469, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.439531
Model 0 best validation binary_cross_entropy = 0.300251 on epoch 194
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.403390
Ensemble test binary_cross_entropy = 0.403390
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.329987
Epoch 1
Validation binary_cross_entropy = 89.897217
Epoch 2
Validation binary_cross_entropy = 3.424388
Epoch 3
Validation binary_cross_entropy = 5.144480
Epoch 4
Loss = 6.6408e+01, PNorm = 60.3415, GNorm = 1316.9902, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 17.381088
Epoch 5
Validation binary_cross_entropy = 30.325808
Epoch 6
Validation binary_cross_entropy = 14.557586
Epoch 7
Validation binary_cross_entropy = 12.658442
Epoch 8
Validation binary_cross_entropy = 23.885502
Epoch 9
Loss = 8.7960e+01, PNorm = 60.4754, GNorm = 1311.7071, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 15.355352
Epoch 10
Validation binary_cross_entropy = 6.318293
Epoch 11
Validation binary_cross_entropy = 6.582220
Epoch 12
Validation binary_cross_entropy = 22.534351
Epoch 13
Validation binary_cross_entropy = 13.745848
Epoch 14
Loss = 3.4611e+01, PNorm = 60.6428, GNorm = 1148.2981, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 5.701260
Epoch 15
Validation binary_cross_entropy = 5.488584
Epoch 16
Validation binary_cross_entropy = 6.343360
Epoch 17
Validation binary_cross_entropy = 6.895384
Epoch 18
Validation binary_cross_entropy = 7.764972
Epoch 19
Loss = 4.2134e+01, PNorm = 60.7986, GNorm = 572.2750, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 28.612917
Epoch 20
Validation binary_cross_entropy = 6.712437
Epoch 21
Validation binary_cross_entropy = 5.916333
Epoch 22
Validation binary_cross_entropy = 6.691182
Epoch 23
Validation binary_cross_entropy = 14.999865
Epoch 24
Loss = 4.9925e+01, PNorm = 60.9343, GNorm = 876.4096, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 10.423956
Epoch 25
Validation binary_cross_entropy = 12.337134
Epoch 26
Validation binary_cross_entropy = 17.739599
Epoch 27
Validation binary_cross_entropy = 16.314693
Epoch 28
Validation binary_cross_entropy = 7.783000
Epoch 29
Loss = 4.2362e+01, PNorm = 61.0578, GNorm = 932.7763, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.934451
Epoch 30
Validation binary_cross_entropy = 8.677959
Epoch 31
Validation binary_cross_entropy = 38.978168
Epoch 32
Validation binary_cross_entropy = 59.152462
Epoch 33
Validation binary_cross_entropy = 14.786493
Epoch 34
Loss = 4.0310e+01, PNorm = 61.1336, GNorm = 896.6124, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.677295
Epoch 35
Validation binary_cross_entropy = 9.242393
Epoch 36
Validation binary_cross_entropy = 17.850548
Epoch 37
Validation binary_cross_entropy = 20.526836
Epoch 38
Validation binary_cross_entropy = 16.116079
Epoch 39
Loss = 3.1546e+01, PNorm = 61.1984, GNorm = 653.7124, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 14.378592
Epoch 40
Validation binary_cross_entropy = 25.959370
Epoch 41
Validation binary_cross_entropy = 29.503994
Epoch 42
Validation binary_cross_entropy = 20.073452
Epoch 43
Validation binary_cross_entropy = 10.200181
Epoch 44
Loss = 2.8603e+01, PNorm = 61.2595, GNorm = 582.8092, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 8.000551
Epoch 45
Validation binary_cross_entropy = 14.562839
Epoch 46
Validation binary_cross_entropy = 20.472073
Epoch 47
Validation binary_cross_entropy = 11.230976
Epoch 48
Validation binary_cross_entropy = 5.641060
Epoch 49
Loss = 3.5899e+01, PNorm = 61.3058, GNorm = 692.2224, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 7.493799
Epoch 50
Validation binary_cross_entropy = 12.007041
Epoch 51
Validation binary_cross_entropy = 26.593508
Epoch 52
Validation binary_cross_entropy = 22.622234
Epoch 53
Validation binary_cross_entropy = 11.728273
Epoch 54
Loss = 8.9781e+00, PNorm = 61.3407, GNorm = 399.8218, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 5.991309
Epoch 55
Validation binary_cross_entropy = 4.956662
Epoch 56
Validation binary_cross_entropy = 4.895059
Epoch 57
Validation binary_cross_entropy = 4.942577
Epoch 58
Validation binary_cross_entropy = 5.037651
Epoch 59
Loss = 2.6966e+01, PNorm = 61.3841, GNorm = 330.3639, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 2.556700
Epoch 60
Validation binary_cross_entropy = 2.806051
Epoch 61
Validation binary_cross_entropy = 9.103025
Epoch 62
Validation binary_cross_entropy = 10.259834
Epoch 63
Validation binary_cross_entropy = 5.021943
Epoch 64
Loss = 1.2913e+01, PNorm = 61.4241, GNorm = 316.9733, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 4.684919
Epoch 65
Validation binary_cross_entropy = 2.294037
Epoch 66
Validation binary_cross_entropy = 1.905987
Epoch 67
Validation binary_cross_entropy = 2.299001
Epoch 68
Validation binary_cross_entropy = 3.931220
Epoch 69
Loss = 9.6135e+00, PNorm = 61.4912, GNorm = 424.5391, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 4.510137
Epoch 70
Validation binary_cross_entropy = 2.607592
Epoch 71
Validation binary_cross_entropy = 1.628032
Epoch 72
Validation binary_cross_entropy = 3.251118
Epoch 73
Validation binary_cross_entropy = 5.565469
Epoch 74
Loss = 9.1430e+00, PNorm = 61.5516, GNorm = 306.9948, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 5.678823
Epoch 75
Validation binary_cross_entropy = 4.114636
Epoch 76
Validation binary_cross_entropy = 1.339937
Epoch 77
Validation binary_cross_entropy = 1.802582
Epoch 78
Validation binary_cross_entropy = 1.692337
Epoch 79
Loss = 1.0417e+01, PNorm = 61.6317, GNorm = 242.9328, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.500661
Epoch 80
Validation binary_cross_entropy = 4.597334
Epoch 81
Validation binary_cross_entropy = 4.081782
Epoch 82
Validation binary_cross_entropy = 3.124982
Epoch 83
Validation binary_cross_entropy = 1.510603
Epoch 84
Loss = 7.3129e+00, PNorm = 61.6941, GNorm = 198.6514, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.611643
Epoch 85
Validation binary_cross_entropy = 0.756422
Epoch 86
Validation binary_cross_entropy = 1.188671
Epoch 87
Validation binary_cross_entropy = 2.932180
Epoch 88
Validation binary_cross_entropy = 6.359521
Epoch 89
Loss = 1.3075e+01, PNorm = 61.7035, GNorm = 559.0766, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 5.388132
Epoch 90
Validation binary_cross_entropy = 2.479831
Epoch 91
Validation binary_cross_entropy = 1.635388
Epoch 92
Validation binary_cross_entropy = 1.703818
Epoch 93
Validation binary_cross_entropy = 1.537112
Epoch 94
Loss = 5.7677e+00, PNorm = 61.7242, GNorm = 237.9686, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.418888
Epoch 95
Validation binary_cross_entropy = 1.187818
Epoch 96
Validation binary_cross_entropy = 1.150030
Epoch 97
Validation binary_cross_entropy = 0.649795
Epoch 98
Validation binary_cross_entropy = 0.635450
Epoch 99
Loss = 1.4052e+01, PNorm = 61.7609, GNorm = 362.6315, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.617393
Epoch 100
Validation binary_cross_entropy = 0.578064
Epoch 101
Validation binary_cross_entropy = 0.563321
Epoch 102
Validation binary_cross_entropy = 0.586211
Epoch 103
Validation binary_cross_entropy = 0.647484
Epoch 104
Loss = 9.2814e+00, PNorm = 61.7993, GNorm = 155.8387, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.550384
Epoch 105
Validation binary_cross_entropy = 0.559989
Epoch 106
Validation binary_cross_entropy = 0.682104
Epoch 107
Validation binary_cross_entropy = 0.764380
Epoch 108
Validation binary_cross_entropy = 0.567515
Epoch 109
Loss = 7.3509e+00, PNorm = 61.8236, GNorm = 255.2327, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.479649
Epoch 110
Validation binary_cross_entropy = 0.535416
Epoch 111
Validation binary_cross_entropy = 0.577951
Epoch 112
Validation binary_cross_entropy = 0.544280
Epoch 113
Validation binary_cross_entropy = 0.562663
Epoch 114
Loss = 5.3881e+00, PNorm = 61.8381, GNorm = 161.4578, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.573681
Epoch 115
Validation binary_cross_entropy = 0.560408
Epoch 116
Validation binary_cross_entropy = 0.536105
Epoch 117
Validation binary_cross_entropy = 0.564314
Epoch 118
Validation binary_cross_entropy = 0.587662
Epoch 119
Loss = 3.1403e+00, PNorm = 61.8440, GNorm = 130.7658, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.584164
Epoch 120
Validation binary_cross_entropy = 0.571984
Epoch 121
Validation binary_cross_entropy = 0.560917
Epoch 122
Validation binary_cross_entropy = 0.612019
Epoch 123
Validation binary_cross_entropy = 0.608738
Epoch 124
Loss = 2.8373e+00, PNorm = 61.8679, GNorm = 199.7073, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.586742
Epoch 125
Validation binary_cross_entropy = 0.517595
Epoch 126
Validation binary_cross_entropy = 0.487354
Epoch 127
Validation binary_cross_entropy = 0.645307
Epoch 128
Validation binary_cross_entropy = 0.877388
Epoch 129
Loss = 3.2976e+00, PNorm = 61.8723, GNorm = 219.8122, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.498131
Epoch 130
Validation binary_cross_entropy = 0.404845
Epoch 131
Validation binary_cross_entropy = 0.447256
Epoch 132
Validation binary_cross_entropy = 0.536417
Epoch 133
Validation binary_cross_entropy = 0.434220
Epoch 134
Loss = 4.1169e+00, PNorm = 61.8790, GNorm = 135.5024, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.373773
Epoch 135
Validation binary_cross_entropy = 0.422659
Epoch 136
Validation binary_cross_entropy = 0.463585
Epoch 137
Validation binary_cross_entropy = 0.564574
Epoch 138
Validation binary_cross_entropy = 0.521680
Epoch 139
Loss = 5.1052e+00, PNorm = 61.8925, GNorm = 154.6346, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.438346
Epoch 140
Validation binary_cross_entropy = 0.423264
Epoch 141
Validation binary_cross_entropy = 0.427237
Epoch 142
Validation binary_cross_entropy = 0.420532
Epoch 143
Validation binary_cross_entropy = 0.418901
Epoch 144
Loss = 5.8548e+00, PNorm = 61.9171, GNorm = 183.9298, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.518076
Epoch 145
Validation binary_cross_entropy = 0.704125
Epoch 146
Validation binary_cross_entropy = 1.547456
Epoch 147
Validation binary_cross_entropy = 1.697101
Epoch 148
Validation binary_cross_entropy = 1.550096
Epoch 149
Loss = 3.3615e+00, PNorm = 61.9409, GNorm = 170.7462, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.203247
Epoch 150
Validation binary_cross_entropy = 0.420642
Epoch 151
Validation binary_cross_entropy = 0.391973
Epoch 152
Validation binary_cross_entropy = 0.376986
Epoch 153
Validation binary_cross_entropy = 0.365124
Epoch 154
Loss = 3.8704e+00, PNorm = 61.9608, GNorm = 106.6020, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.343862
Epoch 155
Validation binary_cross_entropy = 0.338890
Epoch 156
Validation binary_cross_entropy = 0.337472
Epoch 157
Validation binary_cross_entropy = 0.336774
Epoch 158
Validation binary_cross_entropy = 0.349859
Epoch 159
Loss = 3.4951e+00, PNorm = 61.9750, GNorm = 136.9323, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.328490
Epoch 160
Validation binary_cross_entropy = 0.321421
Epoch 161
Validation binary_cross_entropy = 0.341018
Epoch 162
Validation binary_cross_entropy = 0.379988
Epoch 163
Validation binary_cross_entropy = 0.372317
Epoch 164
Loss = 3.3771e+00, PNorm = 61.9854, GNorm = 163.5911, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.342760
Epoch 165
Validation binary_cross_entropy = 0.308652
Epoch 166
Validation binary_cross_entropy = 0.308381
Epoch 167
Validation binary_cross_entropy = 0.314376
Epoch 168
Validation binary_cross_entropy = 0.328461
Epoch 169
Loss = 2.4543e+00, PNorm = 62.0019, GNorm = 74.0339, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.369901
Epoch 170
Validation binary_cross_entropy = 0.539182
Epoch 171
Validation binary_cross_entropy = 0.860551
Epoch 172
Validation binary_cross_entropy = 1.065111
Epoch 173
Validation binary_cross_entropy = 0.915717
Epoch 174
Loss = 2.9734e+00, PNorm = 62.0103, GNorm = 109.8943, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.637092
Epoch 175
Validation binary_cross_entropy = 0.485010
Epoch 176
Validation binary_cross_entropy = 0.417174
Epoch 177
Validation binary_cross_entropy = 0.404156
Epoch 178
Validation binary_cross_entropy = 0.419381
Epoch 179
Loss = 3.5830e+00, PNorm = 62.0151, GNorm = 150.4667, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.471814
Epoch 180
Validation binary_cross_entropy = 0.548568
Epoch 181
Validation binary_cross_entropy = 0.613892
Epoch 182
Validation binary_cross_entropy = 0.670413
Epoch 183
Validation binary_cross_entropy = 0.723176
Epoch 184
Loss = 2.4964e+00, PNorm = 62.0302, GNorm = 133.9713, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.761164
Epoch 185
Validation binary_cross_entropy = 0.711361
Epoch 186
Validation binary_cross_entropy = 0.556202
Epoch 187
Validation binary_cross_entropy = 0.453544
Epoch 188
Validation binary_cross_entropy = 0.428854
Epoch 189
Loss = 1.7708e+00, PNorm = 62.0608, GNorm = 95.9729, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.446430
Epoch 190
Validation binary_cross_entropy = 0.511049
Epoch 191
Validation binary_cross_entropy = 0.590025
Epoch 192
Validation binary_cross_entropy = 0.719993
Epoch 193
Validation binary_cross_entropy = 0.734210
Epoch 194
Loss = 2.0152e+00, PNorm = 62.0868, GNorm = 108.1828, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.665219
Epoch 195
Validation binary_cross_entropy = 0.624740
Epoch 196
Validation binary_cross_entropy = 0.561560
Epoch 197
Validation binary_cross_entropy = 0.535291
Epoch 198
Validation binary_cross_entropy = 0.542959
Epoch 199
Loss = 1.6000e+00, PNorm = 62.0927, GNorm = 65.8060, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.510310
Epoch 200
Validation binary_cross_entropy = 0.471186
Epoch 201
Validation binary_cross_entropy = 0.432160
Epoch 202
Validation binary_cross_entropy = 0.408252
Epoch 203
Validation binary_cross_entropy = 0.383162
Epoch 204
Loss = 1.8608e+00, PNorm = 62.1008, GNorm = 109.5939, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.392808
Epoch 205
Validation binary_cross_entropy = 0.399065
Epoch 206
Validation binary_cross_entropy = 0.419133
Epoch 207
Validation binary_cross_entropy = 0.446548
Epoch 208
Validation binary_cross_entropy = 0.455855
Epoch 209
Loss = 7.7859e-01, PNorm = 62.1099, GNorm = 68.0068, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.462898
Epoch 210
Validation binary_cross_entropy = 0.462257
Epoch 211
Validation binary_cross_entropy = 0.504208
Epoch 212
Validation binary_cross_entropy = 0.542035
Epoch 213
Validation binary_cross_entropy = 0.556891
Epoch 214
Loss = 1.1896e+00, PNorm = 62.1290, GNorm = 98.6517, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.528561
Epoch 215
Validation binary_cross_entropy = 0.466776
Epoch 216
Validation binary_cross_entropy = 0.407683
Epoch 217
Validation binary_cross_entropy = 0.363367
Epoch 218
Validation binary_cross_entropy = 0.343344
Epoch 219
Loss = 1.3495e+00, PNorm = 62.1366, GNorm = 68.8268, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.346521
Epoch 220
Validation binary_cross_entropy = 0.362153
Epoch 221
Validation binary_cross_entropy = 0.397083
Epoch 222
Validation binary_cross_entropy = 0.441696
Epoch 223
Validation binary_cross_entropy = 0.492359
Epoch 224
Loss = 1.1578e+00, PNorm = 62.1492, GNorm = 61.0036, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.536196
Epoch 225
Validation binary_cross_entropy = 0.575506
Epoch 226
Validation binary_cross_entropy = 0.574804
Epoch 227
Validation binary_cross_entropy = 0.574141
Epoch 228
Validation binary_cross_entropy = 0.553591
Epoch 229
Loss = 8.8624e-01, PNorm = 62.1707, GNorm = 83.1167, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.503628
Epoch 230
Validation binary_cross_entropy = 0.460985
Epoch 231
Validation binary_cross_entropy = 0.433842
Epoch 232
Validation binary_cross_entropy = 0.454915
Epoch 233
Validation binary_cross_entropy = 0.497164
Epoch 234
Loss = 7.9663e-01, PNorm = 62.1833, GNorm = 79.1609, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.538111
Epoch 235
Validation binary_cross_entropy = 0.578098
Epoch 236
Validation binary_cross_entropy = 0.584385
Epoch 237
Validation binary_cross_entropy = 0.587522
Epoch 238
Validation binary_cross_entropy = 0.575295
Epoch 239
Loss = 1.7875e+00, PNorm = 62.1776, GNorm = 80.3776, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.549309
Epoch 240
Validation binary_cross_entropy = 0.521311
Epoch 241
Validation binary_cross_entropy = 0.474701
Epoch 242
Validation binary_cross_entropy = 0.456858
Epoch 243
Validation binary_cross_entropy = 0.435248
Epoch 244
Loss = 1.3519e+00, PNorm = 62.1864, GNorm = 91.6429, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.440541
Epoch 245
Validation binary_cross_entropy = 0.450919
Epoch 246
Validation binary_cross_entropy = 0.440939
Epoch 247
Validation binary_cross_entropy = 0.441296
Epoch 248
Validation binary_cross_entropy = 0.444092
Epoch 249
Loss = 9.9942e-01, PNorm = 62.2004, GNorm = 73.1304, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.463895
Epoch 250
Validation binary_cross_entropy = 0.486378
Epoch 251
Validation binary_cross_entropy = 0.486837
Epoch 252
Validation binary_cross_entropy = 0.488680
Epoch 253
Validation binary_cross_entropy = 0.486721
Epoch 254
Loss = 9.7264e-01, PNorm = 62.2152, GNorm = 94.5215, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.470641
Epoch 255
Validation binary_cross_entropy = 0.449702
Epoch 256
Validation binary_cross_entropy = 0.421722
Epoch 257
Validation binary_cross_entropy = 0.411410
Epoch 258
Validation binary_cross_entropy = 0.411367
Epoch 259
Loss = 7.6213e-01, PNorm = 62.2330, GNorm = 49.2242, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.423990
Epoch 260
Validation binary_cross_entropy = 0.448150
Epoch 261
Validation binary_cross_entropy = 0.481060
Epoch 262
Validation binary_cross_entropy = 0.481149
Epoch 263
Validation binary_cross_entropy = 0.489596
Epoch 264
Loss = 9.6473e-01, PNorm = 62.2307, GNorm = 44.0206, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.492737
Epoch 265
Validation binary_cross_entropy = 0.478347
Epoch 266
Validation binary_cross_entropy = 0.458619
Epoch 267
Validation binary_cross_entropy = 0.437248
Epoch 268
Validation binary_cross_entropy = 0.436945
Epoch 269
Loss = 1.0222e+00, PNorm = 62.2373, GNorm = 82.6864, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.451752
Epoch 270
Validation binary_cross_entropy = 0.463906
Epoch 271
Validation binary_cross_entropy = 0.492544
Epoch 272
Validation binary_cross_entropy = 0.510932
Epoch 273
Validation binary_cross_entropy = 0.535249
Epoch 274
Loss = 7.8793e-01, PNorm = 62.2410, GNorm = 71.0804, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.571296
Epoch 275
Validation binary_cross_entropy = 0.613220
Epoch 276
Validation binary_cross_entropy = 0.640082
Epoch 277
Validation binary_cross_entropy = 0.642053
Epoch 278
Validation binary_cross_entropy = 0.608887
Epoch 279
Loss = 9.8042e-01, PNorm = 62.2480, GNorm = 88.3552, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.575566
Epoch 280
Validation binary_cross_entropy = 0.563876
Epoch 281
Validation binary_cross_entropy = 0.538748
Epoch 282
Validation binary_cross_entropy = 0.504190
Epoch 283
Validation binary_cross_entropy = 0.474212
Epoch 284
Loss = 7.8182e-01, PNorm = 62.2667, GNorm = 59.0428, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.450913
Epoch 285
Validation binary_cross_entropy = 0.458802
Epoch 286
Validation binary_cross_entropy = 0.481909
Epoch 287
Validation binary_cross_entropy = 0.510761
Epoch 288
Validation binary_cross_entropy = 0.542816
Epoch 289
Loss = 1.6201e+00, PNorm = 62.2710, GNorm = 108.1565, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.571675
Epoch 290
Validation binary_cross_entropy = 0.587766
Epoch 291
Validation binary_cross_entropy = 0.594299
Epoch 292
Validation binary_cross_entropy = 0.602459
Epoch 293
Validation binary_cross_entropy = 0.611606
Epoch 294
Loss = 8.8964e-01, PNorm = 62.2848, GNorm = 48.0788, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.610680
Epoch 295
Validation binary_cross_entropy = 0.603142
Epoch 296
Validation binary_cross_entropy = 0.601067
Epoch 297
Validation binary_cross_entropy = 0.569801
Epoch 298
Validation binary_cross_entropy = 0.523106
Epoch 299
Loss = 1.0026e+00, PNorm = 62.3014, GNorm = 106.4518, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.482131
Model 0 best validation binary_cross_entropy = 0.308381 on epoch 166
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.396205
Ensemble test binary_cross_entropy = 0.396205
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.302307
Epoch 1
Validation binary_cross_entropy = 85.871368
Epoch 2
Validation binary_cross_entropy = 5.509556
Epoch 3
Validation binary_cross_entropy = 5.761198
Epoch 4
Loss = 8.4016e+01, PNorm = 60.3414, GNorm = 1318.3567, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 23.689331
Epoch 5
Validation binary_cross_entropy = 38.037312
Epoch 6
Validation binary_cross_entropy = 7.733164
Epoch 7
Validation binary_cross_entropy = 6.794150
Epoch 8
Validation binary_cross_entropy = 20.349909
Epoch 9
Loss = 5.3453e+01, PNorm = 60.4694, GNorm = 1106.2708, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 34.371437
Epoch 10
Validation binary_cross_entropy = 6.147722
Epoch 11
Validation binary_cross_entropy = 6.161102
Epoch 12
Validation binary_cross_entropy = 20.895512
Epoch 13
Validation binary_cross_entropy = 29.007732
Epoch 14
Loss = 6.2748e+01, PNorm = 60.6208, GNorm = 788.5009, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 6.061522
Epoch 15
Validation binary_cross_entropy = 4.929449
Epoch 16
Validation binary_cross_entropy = 53.912842
Epoch 17
Validation binary_cross_entropy = 28.571604
Epoch 18
Validation binary_cross_entropy = 4.486094
Epoch 19
Loss = 4.7462e+01, PNorm = 60.7643, GNorm = 594.8761, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 3.936529
Epoch 20
Validation binary_cross_entropy = 27.408321
Epoch 21
Validation binary_cross_entropy = 55.687885
Epoch 22
Validation binary_cross_entropy = 30.644367
Epoch 23
Validation binary_cross_entropy = 4.859294
Epoch 24
Loss = 6.4382e+01, PNorm = 60.8983, GNorm = 1857.7416, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 4.348840
Epoch 25
Validation binary_cross_entropy = 28.559851
Epoch 26
Validation binary_cross_entropy = 57.850761
Epoch 27
Validation binary_cross_entropy = 32.260094
Epoch 28
Validation binary_cross_entropy = 5.500970
Epoch 29
Loss = 5.0437e+01, PNorm = 60.9756, GNorm = 691.4882, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 5.281184
Epoch 30
Validation binary_cross_entropy = 5.739410
Epoch 31
Validation binary_cross_entropy = 14.196499
Epoch 32
Validation binary_cross_entropy = 29.717577
Epoch 33
Validation binary_cross_entropy = 31.374947
Epoch 34
Loss = 3.1648e+01, PNorm = 61.0621, GNorm = 764.6697, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 18.900120
Epoch 35
Validation binary_cross_entropy = 5.024288
Epoch 36
Validation binary_cross_entropy = 3.325649
Epoch 37
Validation binary_cross_entropy = 5.344804
Epoch 38
Validation binary_cross_entropy = 17.754992
Epoch 39
Loss = 2.9231e+01, PNorm = 61.1152, GNorm = 576.9935, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 28.283848
Epoch 40
Validation binary_cross_entropy = 17.825823
Epoch 41
Validation binary_cross_entropy = 5.384800
Epoch 42
Validation binary_cross_entropy = 3.255459
Epoch 43
Validation binary_cross_entropy = 3.290611
Epoch 44
Loss = 1.8601e+01, PNorm = 61.1473, GNorm = 723.9199, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 2.969367
Epoch 45
Validation binary_cross_entropy = 4.901872
Epoch 46
Validation binary_cross_entropy = 14.074861
Epoch 47
Validation binary_cross_entropy = 16.368082
Epoch 48
Validation binary_cross_entropy = 12.317196
Epoch 49
Loss = 1.3838e+01, PNorm = 61.1715, GNorm = 363.2229, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.930998
Epoch 50
Validation binary_cross_entropy = 3.067425
Epoch 51
Validation binary_cross_entropy = 2.600373
Epoch 52
Validation binary_cross_entropy = 4.682640
Epoch 53
Validation binary_cross_entropy = 13.049377
Epoch 54
Loss = 1.6529e+01, PNorm = 61.1944, GNorm = 473.8189, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 19.055475
Epoch 55
Validation binary_cross_entropy = 11.294271
Epoch 56
Validation binary_cross_entropy = 2.946258
Epoch 57
Validation binary_cross_entropy = 2.689296
Epoch 58
Validation binary_cross_entropy = 2.431777
Epoch 59
Loss = 1.6363e+01, PNorm = 61.2167, GNorm = 420.0413, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 3.159987
Epoch 60
Validation binary_cross_entropy = 11.587168
Epoch 61
Validation binary_cross_entropy = 13.353466
Epoch 62
Validation binary_cross_entropy = 5.180403
Epoch 63
Validation binary_cross_entropy = 3.505661
Epoch 64
Loss = 9.4819e+00, PNorm = 61.2395, GNorm = 318.1082, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 4.054636
Epoch 65
Validation binary_cross_entropy = 4.940441
Epoch 66
Validation binary_cross_entropy = 8.216678
Epoch 67
Validation binary_cross_entropy = 6.950156
Epoch 68
Validation binary_cross_entropy = 5.743691
Epoch 69
Loss = 7.3300e+00, PNorm = 61.2637, GNorm = 170.8997, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 7.145385
Epoch 70
Validation binary_cross_entropy = 11.632283
Epoch 71
Validation binary_cross_entropy = 12.936481
Epoch 72
Validation binary_cross_entropy = 7.636683
Epoch 73
Validation binary_cross_entropy = 3.130225
Epoch 74
Loss = 9.7542e+00, PNorm = 61.2990, GNorm = 356.7502, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 3.008710
Epoch 75
Validation binary_cross_entropy = 3.007905
Epoch 76
Validation binary_cross_entropy = 3.587499
Epoch 77
Validation binary_cross_entropy = 5.364332
Epoch 78
Validation binary_cross_entropy = 8.706917
Epoch 79
Loss = 6.5030e+00, PNorm = 61.3606, GNorm = 364.2968, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 12.697392
Epoch 80
Validation binary_cross_entropy = 9.679651
Epoch 81
Validation binary_cross_entropy = 5.110741
Epoch 82
Validation binary_cross_entropy = 1.825974
Epoch 83
Validation binary_cross_entropy = 2.229310
Epoch 84
Loss = 9.1947e+00, PNorm = 61.4432, GNorm = 345.5323, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 8.234108
Epoch 85
Validation binary_cross_entropy = 14.458602
Epoch 86
Validation binary_cross_entropy = 9.119720
Epoch 87
Validation binary_cross_entropy = 2.971024
Epoch 88
Validation binary_cross_entropy = 1.734176
Epoch 89
Loss = 1.5101e+01, PNorm = 61.5220, GNorm = 361.2965, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.420109
Epoch 90
Validation binary_cross_entropy = 1.669945
Epoch 91
Validation binary_cross_entropy = 1.646058
Epoch 92
Validation binary_cross_entropy = 3.283333
Epoch 93
Validation binary_cross_entropy = 3.744851
Epoch 94
Loss = 6.9198e+00, PNorm = 61.5782, GNorm = 283.7354, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 3.534218
Epoch 95
Validation binary_cross_entropy = 1.971303
Epoch 96
Validation binary_cross_entropy = 1.135999
Epoch 97
Validation binary_cross_entropy = 1.005668
Epoch 98
Validation binary_cross_entropy = 0.927040
Epoch 99
Loss = 9.3218e+00, PNorm = 61.6341, GNorm = 200.5397, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.846765
Epoch 100
Validation binary_cross_entropy = 0.767705
Epoch 101
Validation binary_cross_entropy = 0.751149
Epoch 102
Validation binary_cross_entropy = 0.853659
Epoch 103
Validation binary_cross_entropy = 0.889928
Epoch 104
Loss = 5.7660e+00, PNorm = 61.6871, GNorm = 218.5915, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.809153
Epoch 105
Validation binary_cross_entropy = 0.714985
Epoch 106
Validation binary_cross_entropy = 0.687157
Epoch 107
Validation binary_cross_entropy = 0.777087
Epoch 108
Validation binary_cross_entropy = 1.046178
Epoch 109
Loss = 6.1693e+00, PNorm = 61.7258, GNorm = 197.0812, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.573054
Epoch 110
Validation binary_cross_entropy = 1.998929
Epoch 111
Validation binary_cross_entropy = 1.009880
Epoch 112
Validation binary_cross_entropy = 0.812655
Epoch 113
Validation binary_cross_entropy = 0.879425
Epoch 114
Loss = 8.0190e+00, PNorm = 61.7447, GNorm = 241.3709, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.852683
Epoch 115
Validation binary_cross_entropy = 0.977717
Epoch 116
Validation binary_cross_entropy = 1.563874
Epoch 117
Validation binary_cross_entropy = 1.001111
Epoch 118
Validation binary_cross_entropy = 0.933093
Epoch 119
Loss = 4.8745e+00, PNorm = 61.7651, GNorm = 156.1300, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.990493
Epoch 120
Validation binary_cross_entropy = 0.799871
Epoch 121
Validation binary_cross_entropy = 0.586436
Epoch 122
Validation binary_cross_entropy = 0.581095
Epoch 123
Validation binary_cross_entropy = 0.610038
Epoch 124
Loss = 5.5800e+00, PNorm = 61.7850, GNorm = 186.7919, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.613781
Epoch 125
Validation binary_cross_entropy = 0.672896
Epoch 126
Validation binary_cross_entropy = 0.698653
Epoch 127
Validation binary_cross_entropy = 0.639088
Epoch 128
Validation binary_cross_entropy = 0.532935
Epoch 129
Loss = 4.4855e+00, PNorm = 61.8011, GNorm = 185.7207, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.546399
Epoch 130
Validation binary_cross_entropy = 0.591893
Epoch 131
Validation binary_cross_entropy = 0.613827
Epoch 132
Validation binary_cross_entropy = 0.567293
Epoch 133
Validation binary_cross_entropy = 0.549756
Epoch 134
Loss = 5.8644e+00, PNorm = 61.8113, GNorm = 146.2843, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.526868
Epoch 135
Validation binary_cross_entropy = 0.512850
Epoch 136
Validation binary_cross_entropy = 0.503476
Epoch 137
Validation binary_cross_entropy = 0.464711
Epoch 138
Validation binary_cross_entropy = 0.477014
Epoch 139
Loss = 4.5131e+00, PNorm = 61.8271, GNorm = 156.7568, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.493429
Epoch 140
Validation binary_cross_entropy = 0.495429
Epoch 141
Validation binary_cross_entropy = 0.499434
Epoch 142
Validation binary_cross_entropy = 0.456598
Epoch 143
Validation binary_cross_entropy = 0.413346
Epoch 144
Loss = 4.2949e+00, PNorm = 61.8434, GNorm = 156.8191, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.424956
Epoch 145
Validation binary_cross_entropy = 0.428255
Epoch 146
Validation binary_cross_entropy = 0.415651
Epoch 147
Validation binary_cross_entropy = 0.433655
Epoch 148
Validation binary_cross_entropy = 0.496707
Epoch 149
Loss = 5.1154e+00, PNorm = 61.8639, GNorm = 193.5262, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.553824
Epoch 150
Validation binary_cross_entropy = 0.508676
Epoch 151
Validation binary_cross_entropy = 0.500824
Epoch 152
Validation binary_cross_entropy = 0.474784
Epoch 153
Validation binary_cross_entropy = 0.419861
Epoch 154
Loss = 3.9498e+00, PNorm = 61.8839, GNorm = 139.4375, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.403752
Epoch 155
Validation binary_cross_entropy = 0.440132
Epoch 156
Validation binary_cross_entropy = 0.453786
Epoch 157
Validation binary_cross_entropy = 0.517184
Epoch 158
Validation binary_cross_entropy = 0.580719
Epoch 159
Loss = 4.6346e+00, PNorm = 61.8875, GNorm = 183.8711, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.596121
Epoch 160
Validation binary_cross_entropy = 0.577157
Epoch 161
Validation binary_cross_entropy = 0.566478
Epoch 162
Validation binary_cross_entropy = 0.501481
Epoch 163
Validation binary_cross_entropy = 0.477152
Epoch 164
Loss = 2.2454e+00, PNorm = 61.8982, GNorm = 151.3023, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.451490
Epoch 165
Validation binary_cross_entropy = 0.403767
Epoch 166
Validation binary_cross_entropy = 0.409028
Epoch 167
Validation binary_cross_entropy = 0.410479
Epoch 168
Validation binary_cross_entropy = 0.432478
Epoch 169
Loss = 1.7919e+00, PNorm = 61.9107, GNorm = 115.4831, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.461818
Epoch 170
Validation binary_cross_entropy = 0.501575
Epoch 171
Validation binary_cross_entropy = 0.528221
Epoch 172
Validation binary_cross_entropy = 0.528104
Epoch 173
Validation binary_cross_entropy = 0.507051
Epoch 174
Loss = 2.7845e+00, PNorm = 61.9280, GNorm = 144.1157, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.482422
Epoch 175
Validation binary_cross_entropy = 0.459028
Epoch 176
Validation binary_cross_entropy = 0.469116
Epoch 177
Validation binary_cross_entropy = 0.480712
Epoch 178
Validation binary_cross_entropy = 0.485234
Epoch 179
Loss = 2.4612e+00, PNorm = 61.9395, GNorm = 158.0959, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.511914
Epoch 180
Validation binary_cross_entropy = 0.524277
Epoch 181
Validation binary_cross_entropy = 0.574407
Epoch 182
Validation binary_cross_entropy = 0.551942
Epoch 183
Validation binary_cross_entropy = 0.538682
Epoch 184
Loss = 1.6234e+00, PNorm = 61.9511, GNorm = 105.4353, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.534592
Epoch 185
Validation binary_cross_entropy = 0.503818
Epoch 186
Validation binary_cross_entropy = 0.461939
Epoch 187
Validation binary_cross_entropy = 0.433972
Epoch 188
Validation binary_cross_entropy = 0.381840
Epoch 189
Loss = 9.7576e-01, PNorm = 61.9567, GNorm = 54.2151, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.356083
Epoch 190
Validation binary_cross_entropy = 0.344670
Epoch 191
Validation binary_cross_entropy = 0.344243
Epoch 192
Validation binary_cross_entropy = 0.351232
Epoch 193
Validation binary_cross_entropy = 0.371453
Epoch 194
Loss = 1.5461e+00, PNorm = 61.9654, GNorm = 69.9377, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.389861
Epoch 195
Validation binary_cross_entropy = 0.399076
Epoch 196
Validation binary_cross_entropy = 0.379393
Epoch 197
Validation binary_cross_entropy = 0.378677
Epoch 198
Validation binary_cross_entropy = 0.362277
Epoch 199
Loss = 1.1933e+00, PNorm = 61.9833, GNorm = 91.6351, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.348642
Epoch 200
Validation binary_cross_entropy = 0.347578
Epoch 201
Validation binary_cross_entropy = 0.358141
Epoch 202
Validation binary_cross_entropy = 0.395621
Epoch 203
Validation binary_cross_entropy = 0.436981
Epoch 204
Loss = 2.0125e+00, PNorm = 61.9904, GNorm = 105.0905, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.454944
Epoch 205
Validation binary_cross_entropy = 0.485493
Epoch 206
Validation binary_cross_entropy = 0.514084
Epoch 207
Validation binary_cross_entropy = 0.544785
Epoch 208
Validation binary_cross_entropy = 0.560318
Epoch 209
Loss = 1.5497e+00, PNorm = 62.0027, GNorm = 87.2732, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.569205
Epoch 210
Validation binary_cross_entropy = 0.546352
Epoch 211
Validation binary_cross_entropy = 0.517394
Epoch 212
Validation binary_cross_entropy = 0.502592
Epoch 213
Validation binary_cross_entropy = 0.474080
Epoch 214
Loss = 2.1495e+00, PNorm = 62.0129, GNorm = 94.4657, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.447047
Epoch 215
Validation binary_cross_entropy = 0.425336
Epoch 216
Validation binary_cross_entropy = 0.413319
Epoch 217
Validation binary_cross_entropy = 0.408765
Epoch 218
Validation binary_cross_entropy = 0.409878
Epoch 219
Loss = 1.8515e+00, PNorm = 62.0381, GNorm = 97.9973, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.408720
Epoch 220
Validation binary_cross_entropy = 0.444660
Epoch 221
Validation binary_cross_entropy = 0.492330
Epoch 222
Validation binary_cross_entropy = 0.533480
Epoch 223
Validation binary_cross_entropy = 0.549544
Epoch 224
Loss = 1.7636e+00, PNorm = 62.0318, GNorm = 66.5682, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.545616
Epoch 225
Validation binary_cross_entropy = 0.528898
Epoch 226
Validation binary_cross_entropy = 0.490268
Epoch 227
Validation binary_cross_entropy = 0.435991
Epoch 228
Validation binary_cross_entropy = 0.401954
Epoch 229
Loss = 1.8409e+00, PNorm = 62.0381, GNorm = 83.5147, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.392816
Epoch 230
Validation binary_cross_entropy = 0.402067
Epoch 231
Validation binary_cross_entropy = 0.436415
Epoch 232
Validation binary_cross_entropy = 0.475980
Epoch 233
Validation binary_cross_entropy = 0.506415
Epoch 234
Loss = 2.0241e+00, PNorm = 62.0576, GNorm = 113.7075, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.537397
Epoch 235
Validation binary_cross_entropy = 0.559386
Epoch 236
Validation binary_cross_entropy = 0.561320
Epoch 237
Validation binary_cross_entropy = 0.538962
Epoch 238
Validation binary_cross_entropy = 0.507833
Epoch 239
Loss = 1.5681e+00, PNorm = 62.0822, GNorm = 69.3568, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.500080
Epoch 240
Validation binary_cross_entropy = 0.489658
Epoch 241
Validation binary_cross_entropy = 0.491516
Epoch 242
Validation binary_cross_entropy = 0.514015
Epoch 243
Validation binary_cross_entropy = 0.514726
Epoch 244
Loss = 1.0708e+00, PNorm = 62.1175, GNorm = 91.0885, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.515809
Epoch 245
Validation binary_cross_entropy = 0.524853
Epoch 246
Validation binary_cross_entropy = 0.543252
Epoch 247
Validation binary_cross_entropy = 0.554100
Epoch 248
Validation binary_cross_entropy = 0.554564
Epoch 249
Loss = 2.0743e+00, PNorm = 62.1384, GNorm = 98.5889, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.546195
Epoch 250
Validation binary_cross_entropy = 0.543870
Epoch 251
Validation binary_cross_entropy = 0.557464
Epoch 252
Validation binary_cross_entropy = 0.591780
Epoch 253
Validation binary_cross_entropy = 0.626709
Epoch 254
Loss = 1.0042e+00, PNorm = 62.1494, GNorm = 64.6870, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.641228
Epoch 255
Validation binary_cross_entropy = 0.640083
Epoch 256
Validation binary_cross_entropy = 0.635114
Epoch 257
Validation binary_cross_entropy = 0.623196
Epoch 258
Validation binary_cross_entropy = 0.596978
Epoch 259
Loss = 1.3940e+00, PNorm = 62.1506, GNorm = 110.7274, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.559869
Epoch 260
Validation binary_cross_entropy = 0.535798
Epoch 261
Validation binary_cross_entropy = 0.529020
Epoch 262
Validation binary_cross_entropy = 0.528025
Epoch 263
Validation binary_cross_entropy = 0.539274
Epoch 264
Loss = 1.1580e+00, PNorm = 62.1645, GNorm = 79.5610, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.570255
Epoch 265
Validation binary_cross_entropy = 0.614646
Epoch 266
Validation binary_cross_entropy = 0.642789
Epoch 267
Validation binary_cross_entropy = 0.653541
Epoch 268
Validation binary_cross_entropy = 0.644584
Epoch 269
Loss = 1.3018e+00, PNorm = 62.1770, GNorm = 67.0053, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.633587
Epoch 270
Validation binary_cross_entropy = 0.615944
Epoch 271
Validation binary_cross_entropy = 0.606482
Epoch 272
Validation binary_cross_entropy = 0.601332
Epoch 273
Validation binary_cross_entropy = 0.604358
Epoch 274
Loss = 6.5381e-01, PNorm = 62.2054, GNorm = 47.7210, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.612549
Epoch 275
Validation binary_cross_entropy = 0.618227
Epoch 276
Validation binary_cross_entropy = 0.619488
Epoch 277
Validation binary_cross_entropy = 0.623094
Epoch 278
Validation binary_cross_entropy = 0.628852
Epoch 279
Loss = 1.0286e+00, PNorm = 62.2122, GNorm = 70.1585, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.628821
Epoch 280
Validation binary_cross_entropy = 0.621767
Epoch 281
Validation binary_cross_entropy = 0.609991
Epoch 282
Validation binary_cross_entropy = 0.602488
Epoch 283
Validation binary_cross_entropy = 0.601076
Epoch 284
Loss = 1.0108e+00, PNorm = 62.2154, GNorm = 38.7273, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.603629
Epoch 285
Validation binary_cross_entropy = 0.605363
Epoch 286
Validation binary_cross_entropy = 0.614930
Epoch 287
Validation binary_cross_entropy = 0.624531
Epoch 288
Validation binary_cross_entropy = 0.623587
Epoch 289
Loss = 6.0977e-01, PNorm = 62.2230, GNorm = 33.0524, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.618356
Epoch 290
Validation binary_cross_entropy = 0.609445
Epoch 291
Validation binary_cross_entropy = 0.603614
Epoch 292
Validation binary_cross_entropy = 0.603842
Epoch 293
Validation binary_cross_entropy = 0.594268
Epoch 294
Loss = 6.4652e-01, PNorm = 62.2415, GNorm = 59.6470, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.588162
Epoch 295
Validation binary_cross_entropy = 0.583595
Epoch 296
Validation binary_cross_entropy = 0.585190
Epoch 297
Validation binary_cross_entropy = 0.589941
Epoch 298
Validation binary_cross_entropy = 0.600544
Epoch 299
Loss = 6.8571e-01, PNorm = 62.2366, GNorm = 33.6674, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.602566
Model 0 best validation binary_cross_entropy = 0.344243 on epoch 191
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.316820
Ensemble test binary_cross_entropy = 0.316820
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.901634
Epoch 1
Validation binary_cross_entropy = 85.365906
Epoch 2
Validation binary_cross_entropy = 5.653494
Epoch 3
Validation binary_cross_entropy = 26.787109
Epoch 4
Loss = 8.0077e+01, PNorm = 60.3424, GNorm = 2167.8399, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 30.903473
Epoch 5
Validation binary_cross_entropy = 10.980107
Epoch 6
Validation binary_cross_entropy = 12.861270
Epoch 7
Validation binary_cross_entropy = 36.103306
Epoch 8
Validation binary_cross_entropy = 29.731184
Epoch 9
Loss = 4.3411e+01, PNorm = 60.4735, GNorm = 1194.5781, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 15.357764
Epoch 10
Validation binary_cross_entropy = 8.288548
Epoch 11
Validation binary_cross_entropy = 18.426746
Epoch 12
Validation binary_cross_entropy = 12.874001
Epoch 13
Validation binary_cross_entropy = 11.100225
Epoch 14
Loss = 4.9328e+01, PNorm = 60.6413, GNorm = 965.6026, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 9.680081
Epoch 15
Validation binary_cross_entropy = 8.779212
Epoch 16
Validation binary_cross_entropy = 11.571193
Epoch 17
Validation binary_cross_entropy = 9.897144
Epoch 18
Validation binary_cross_entropy = 6.047042
Epoch 19
Loss = 5.1757e+01, PNorm = 60.8061, GNorm = 1021.2306, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 6.953742
Epoch 20
Validation binary_cross_entropy = 25.265368
Epoch 21
Validation binary_cross_entropy = 41.906013
Epoch 22
Validation binary_cross_entropy = 6.623501
Epoch 23
Validation binary_cross_entropy = 5.872523
Epoch 24
Loss = 4.8232e+01, PNorm = 60.9426, GNorm = 1124.1679, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 17.596481
Epoch 25
Validation binary_cross_entropy = 37.328743
Epoch 26
Validation binary_cross_entropy = 22.070808
Epoch 27
Validation binary_cross_entropy = 4.592020
Epoch 28
Validation binary_cross_entropy = 4.812485
Epoch 29
Loss = 3.8389e+01, PNorm = 61.0507, GNorm = 1105.7542, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 14.078652
Epoch 30
Validation binary_cross_entropy = 42.758987
Epoch 31
Validation binary_cross_entropy = 31.514160
Epoch 32
Validation binary_cross_entropy = 8.125919
Epoch 33
Validation binary_cross_entropy = 6.595891
Epoch 34
Loss = 2.6567e+01, PNorm = 61.1424, GNorm = 775.8220, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 6.447370
Epoch 35
Validation binary_cross_entropy = 12.201732
Epoch 36
Validation binary_cross_entropy = 24.046450
Epoch 37
Validation binary_cross_entropy = 9.621848
Epoch 38
Validation binary_cross_entropy = 5.739829
Epoch 39
Loss = 3.7909e+01, PNorm = 61.2046, GNorm = 701.2987, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 5.653375
Epoch 40
Validation binary_cross_entropy = 7.665823
Epoch 41
Validation binary_cross_entropy = 14.781948
Epoch 42
Validation binary_cross_entropy = 13.426428
Epoch 43
Validation binary_cross_entropy = 6.461153
Epoch 44
Loss = 1.4165e+01, PNorm = 61.2501, GNorm = 369.4038, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 4.451983
Epoch 45
Validation binary_cross_entropy = 4.585109
Epoch 46
Validation binary_cross_entropy = 4.397497
Epoch 47
Validation binary_cross_entropy = 4.423334
Epoch 48
Validation binary_cross_entropy = 10.069768
Epoch 49
Loss = 2.0481e+01, PNorm = 61.2953, GNorm = 354.7570, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 4.320206
Epoch 50
Validation binary_cross_entropy = 3.739151
Epoch 51
Validation binary_cross_entropy = 3.256734
Epoch 52
Validation binary_cross_entropy = 2.916404
Epoch 53
Validation binary_cross_entropy = 2.642725
Epoch 54
Loss = 1.1760e+01, PNorm = 61.3443, GNorm = 483.4760, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 6.484591
Epoch 55
Validation binary_cross_entropy = 2.713991
Epoch 56
Validation binary_cross_entropy = 1.252330
Epoch 57
Validation binary_cross_entropy = 1.820582
Epoch 58
Validation binary_cross_entropy = 1.185842
Epoch 59
Loss = 1.9737e+01, PNorm = 61.3789, GNorm = 503.3587, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 9.726002
Epoch 60
Validation binary_cross_entropy = 21.673605
Epoch 61
Validation binary_cross_entropy = 13.620462
Epoch 62
Validation binary_cross_entropy = 5.511188
Epoch 63
Validation binary_cross_entropy = 3.825136
Epoch 64
Loss = 8.8038e+00, PNorm = 61.4110, GNorm = 260.5523, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 5.669096
Epoch 65
Validation binary_cross_entropy = 7.613432
Epoch 66
Validation binary_cross_entropy = 6.850851
Epoch 67
Validation binary_cross_entropy = 8.488793
Epoch 68
Validation binary_cross_entropy = 10.747973
Epoch 69
Loss = 9.8218e+00, PNorm = 61.4468, GNorm = 304.3051, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 6.552733
Epoch 70
Validation binary_cross_entropy = 2.790247
Epoch 71
Validation binary_cross_entropy = 1.884601
Epoch 72
Validation binary_cross_entropy = 1.811080
Epoch 73
Validation binary_cross_entropy = 2.032012
Epoch 74
Loss = 7.5351e+00, PNorm = 61.4804, GNorm = 220.6237, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 10.473908
Epoch 75
Validation binary_cross_entropy = 19.476601
Epoch 76
Validation binary_cross_entropy = 3.956821
Epoch 77
Validation binary_cross_entropy = 1.671542
Epoch 78
Validation binary_cross_entropy = 1.113416
Epoch 79
Loss = 8.8003e+00, PNorm = 61.5042, GNorm = 341.6462, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.122698
Epoch 80
Validation binary_cross_entropy = 2.303418
Epoch 81
Validation binary_cross_entropy = 2.811282
Epoch 82
Validation binary_cross_entropy = 3.270866
Epoch 83
Validation binary_cross_entropy = 2.033374
Epoch 84
Loss = 6.1701e+00, PNorm = 61.5396, GNorm = 293.1468, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.874021
Epoch 85
Validation binary_cross_entropy = 0.902679
Epoch 86
Validation binary_cross_entropy = 0.964325
Epoch 87
Validation binary_cross_entropy = 0.861446
Epoch 88
Validation binary_cross_entropy = 1.438388
Epoch 89
Loss = 4.3560e+00, PNorm = 61.5706, GNorm = 199.5790, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 2.214483
Epoch 90
Validation binary_cross_entropy = 2.217749
Epoch 91
Validation binary_cross_entropy = 1.502374
Epoch 92
Validation binary_cross_entropy = 1.447460
Epoch 93
Validation binary_cross_entropy = 1.555194
Epoch 94
Loss = 6.3469e+00, PNorm = 61.6061, GNorm = 512.8932, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.601452
Epoch 95
Validation binary_cross_entropy = 1.165244
Epoch 96
Validation binary_cross_entropy = 1.093860
Epoch 97
Validation binary_cross_entropy = 1.023463
Epoch 98
Validation binary_cross_entropy = 1.092302
Epoch 99
Loss = 7.9541e+00, PNorm = 61.6120, GNorm = 288.0008, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.172821
Epoch 100
Validation binary_cross_entropy = 1.139779
Epoch 101
Validation binary_cross_entropy = 1.346378
Epoch 102
Validation binary_cross_entropy = 1.715790
Epoch 103
Validation binary_cross_entropy = 1.367843
Epoch 104
Loss = 2.8303e+00, PNorm = 61.6276, GNorm = 141.2673, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.241559
Epoch 105
Validation binary_cross_entropy = 1.252490
Epoch 106
Validation binary_cross_entropy = 1.405924
Epoch 107
Validation binary_cross_entropy = 1.739703
Epoch 108
Validation binary_cross_entropy = 2.036556
Epoch 109
Loss = 5.0685e+00, PNorm = 61.6393, GNorm = 359.7941, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.577792
Epoch 110
Validation binary_cross_entropy = 0.808406
Epoch 111
Validation binary_cross_entropy = 0.702584
Epoch 112
Validation binary_cross_entropy = 0.709576
Epoch 113
Validation binary_cross_entropy = 0.702281
Epoch 114
Loss = 8.4086e+00, PNorm = 61.6750, GNorm = 383.8077, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.643363
Epoch 115
Validation binary_cross_entropy = 1.123914
Epoch 116
Validation binary_cross_entropy = 1.317928
Epoch 117
Validation binary_cross_entropy = 0.782723
Epoch 118
Validation binary_cross_entropy = 0.581620
Epoch 119
Loss = 6.2504e+00, PNorm = 61.7245, GNorm = 247.3511, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.814777
Epoch 120
Validation binary_cross_entropy = 0.693507
Epoch 121
Validation binary_cross_entropy = 0.599137
Epoch 122
Validation binary_cross_entropy = 0.666051
Epoch 123
Validation binary_cross_entropy = 1.264039
Epoch 124
Loss = 3.6539e+00, PNorm = 61.7275, GNorm = 143.4775, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.344713
Epoch 125
Validation binary_cross_entropy = 1.279769
Epoch 126
Validation binary_cross_entropy = 0.756879
Epoch 127
Validation binary_cross_entropy = 0.790473
Epoch 128
Validation binary_cross_entropy = 0.744759
Epoch 129
Loss = 4.2930e+00, PNorm = 61.7412, GNorm = 114.7847, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.665341
Epoch 130
Validation binary_cross_entropy = 0.589571
Epoch 131
Validation binary_cross_entropy = 0.541740
Epoch 132
Validation binary_cross_entropy = 0.516584
Epoch 133
Validation binary_cross_entropy = 0.511404
Epoch 134
Loss = 3.2943e+00, PNorm = 61.7610, GNorm = 134.7718, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.518354
Epoch 135
Validation binary_cross_entropy = 0.594505
Epoch 136
Validation binary_cross_entropy = 0.667949
Epoch 137
Validation binary_cross_entropy = 0.682852
Epoch 138
Validation binary_cross_entropy = 0.676486
Epoch 139
Loss = 2.9108e+00, PNorm = 61.7717, GNorm = 176.5390, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.591836
Epoch 140
Validation binary_cross_entropy = 0.504556
Epoch 141
Validation binary_cross_entropy = 0.441676
Epoch 142
Validation binary_cross_entropy = 0.409361
Epoch 143
Validation binary_cross_entropy = 0.406632
Epoch 144
Loss = 3.1385e+00, PNorm = 61.7872, GNorm = 171.4233, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.391792
Epoch 145
Validation binary_cross_entropy = 0.393195
Epoch 146
Validation binary_cross_entropy = 0.400827
Epoch 147
Validation binary_cross_entropy = 0.439352
Epoch 148
Validation binary_cross_entropy = 0.535074
Epoch 149
Loss = 5.1411e+00, PNorm = 61.8073, GNorm = 155.4522, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.502821
Epoch 150
Validation binary_cross_entropy = 0.395935
Epoch 151
Validation binary_cross_entropy = 0.331590
Epoch 152
Validation binary_cross_entropy = 0.428328
Epoch 153
Validation binary_cross_entropy = 0.618115
Epoch 154
Loss = 3.4050e+00, PNorm = 61.8198, GNorm = 202.2440, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.688781
Epoch 155
Validation binary_cross_entropy = 0.595526
Epoch 156
Validation binary_cross_entropy = 0.368021
Epoch 157
Validation binary_cross_entropy = 0.339353
Epoch 158
Validation binary_cross_entropy = 0.359394
Epoch 159
Loss = 3.8167e+00, PNorm = 61.8340, GNorm = 155.2097, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.370700
Epoch 160
Validation binary_cross_entropy = 0.400128
Epoch 161
Validation binary_cross_entropy = 0.415513
Epoch 162
Validation binary_cross_entropy = 0.396225
Epoch 163
Validation binary_cross_entropy = 0.360856
Epoch 164
Loss = 3.6666e+00, PNorm = 61.8624, GNorm = 247.2959, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.353512
Epoch 165
Validation binary_cross_entropy = 0.376223
Epoch 166
Validation binary_cross_entropy = 0.430061
Epoch 167
Validation binary_cross_entropy = 0.458466
Epoch 168
Validation binary_cross_entropy = 0.474095
Epoch 169
Loss = 2.0113e+00, PNorm = 61.8749, GNorm = 98.8546, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.467378
Epoch 170
Validation binary_cross_entropy = 0.478876
Epoch 171
Validation binary_cross_entropy = 0.445561
Epoch 172
Validation binary_cross_entropy = 0.438635
Epoch 173
Validation binary_cross_entropy = 0.429658
Epoch 174
Loss = 3.7002e+00, PNorm = 61.8917, GNorm = 119.8174, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.464650
Epoch 175
Validation binary_cross_entropy = 0.509307
Epoch 176
Validation binary_cross_entropy = 0.516933
Epoch 177
Validation binary_cross_entropy = 0.409700
Epoch 178
Validation binary_cross_entropy = 0.331777
Epoch 179
Loss = 1.4067e+00, PNorm = 61.9152, GNorm = 112.2029, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.322655
Epoch 180
Validation binary_cross_entropy = 0.318769
Epoch 181
Validation binary_cross_entropy = 0.323784
Epoch 182
Validation binary_cross_entropy = 0.328459
Epoch 183
Validation binary_cross_entropy = 0.363983
Epoch 184
Loss = 2.0812e+00, PNorm = 61.9399, GNorm = 141.6315, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.587857
Epoch 185
Validation binary_cross_entropy = 0.906448
Epoch 186
Validation binary_cross_entropy = 0.969146
Epoch 187
Validation binary_cross_entropy = 0.869052
Epoch 188
Validation binary_cross_entropy = 0.687859
Epoch 189
Loss = 1.1859e+00, PNorm = 61.9538, GNorm = 96.9754, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.543105
Epoch 190
Validation binary_cross_entropy = 0.464419
Epoch 191
Validation binary_cross_entropy = 0.437837
Epoch 192
Validation binary_cross_entropy = 0.446580
Epoch 193
Validation binary_cross_entropy = 0.460993
Epoch 194
Loss = 1.8446e+00, PNorm = 61.9758, GNorm = 81.2245, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.483455
Epoch 195
Validation binary_cross_entropy = 0.513978
Epoch 196
Validation binary_cross_entropy = 0.550791
Epoch 197
Validation binary_cross_entropy = 0.589948
Epoch 198
Validation binary_cross_entropy = 0.630825
Epoch 199
Loss = 1.5720e+00, PNorm = 61.9877, GNorm = 57.6949, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.668574
Epoch 200
Validation binary_cross_entropy = 0.639801
Epoch 201
Validation binary_cross_entropy = 0.593496
Epoch 202
Validation binary_cross_entropy = 0.556947
Epoch 203
Validation binary_cross_entropy = 0.528791
Epoch 204
Loss = 1.2838e+00, PNorm = 62.0066, GNorm = 81.6643, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.512857
Epoch 205
Validation binary_cross_entropy = 0.516170
Epoch 206
Validation binary_cross_entropy = 0.533674
Epoch 207
Validation binary_cross_entropy = 0.546026
Epoch 208
Validation binary_cross_entropy = 0.551813
Epoch 209
Loss = 2.0933e+00, PNorm = 62.0148, GNorm = 73.5749, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.556623
Epoch 210
Validation binary_cross_entropy = 0.560143
Epoch 211
Validation binary_cross_entropy = 0.565729
Epoch 212
Validation binary_cross_entropy = 0.575536
Epoch 213
Validation binary_cross_entropy = 0.579529
Epoch 214
Loss = 1.7851e+00, PNorm = 62.0393, GNorm = 120.3520, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.565928
Epoch 215
Validation binary_cross_entropy = 0.523294
Epoch 216
Validation binary_cross_entropy = 0.515199
Epoch 217
Validation binary_cross_entropy = 0.513058
Epoch 218
Validation binary_cross_entropy = 0.501998
Epoch 219
Loss = 1.0217e+00, PNorm = 62.0736, GNorm = 36.2097, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.477182
Epoch 220
Validation binary_cross_entropy = 0.474711
Epoch 221
Validation binary_cross_entropy = 0.479606
Epoch 222
Validation binary_cross_entropy = 0.515069
Epoch 223
Validation binary_cross_entropy = 0.582717
Epoch 224
Loss = 9.7867e-01, PNorm = 62.0907, GNorm = 94.8049, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.631990
Epoch 225
Validation binary_cross_entropy = 0.626898
Epoch 226
Validation binary_cross_entropy = 0.580354
Epoch 227
Validation binary_cross_entropy = 0.540484
Epoch 228
Validation binary_cross_entropy = 0.502143
Epoch 229
Loss = 7.5615e-01, PNorm = 62.1131, GNorm = 45.6178, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.489600
Epoch 230
Validation binary_cross_entropy = 0.501138
Epoch 231
Validation binary_cross_entropy = 0.520534
Epoch 232
Validation binary_cross_entropy = 0.556225
Epoch 233
Validation binary_cross_entropy = 0.617883
Epoch 234
Loss = 9.5900e-01, PNorm = 62.1410, GNorm = 44.7866, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.640016
Epoch 235
Validation binary_cross_entropy = 0.649506
Epoch 236
Validation binary_cross_entropy = 0.660937
Epoch 237
Validation binary_cross_entropy = 0.643781
Epoch 238
Validation binary_cross_entropy = 0.618478
Epoch 239
Loss = 1.0032e+00, PNorm = 62.1474, GNorm = 54.7147, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.619615
Epoch 240
Validation binary_cross_entropy = 0.627167
Epoch 241
Validation binary_cross_entropy = 0.629912
Epoch 242
Validation binary_cross_entropy = 0.634441
Epoch 243
Validation binary_cross_entropy = 0.636957
Epoch 244
Loss = 1.0179e+00, PNorm = 62.1534, GNorm = 69.0323, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.634642
Epoch 245
Validation binary_cross_entropy = 0.629067
Epoch 246
Validation binary_cross_entropy = 0.641302
Epoch 247
Validation binary_cross_entropy = 0.670047
Epoch 248
Validation binary_cross_entropy = 0.723863
Epoch 249
Loss = 8.4542e-01, PNorm = 62.1636, GNorm = 79.9011, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.718189
Epoch 250
Validation binary_cross_entropy = 0.687252
Epoch 251
Validation binary_cross_entropy = 0.649911
Epoch 252
Validation binary_cross_entropy = 0.616496
Epoch 253
Validation binary_cross_entropy = 0.588772
Epoch 254
Loss = 1.7735e+00, PNorm = 62.1758, GNorm = 72.8626, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.565034
Epoch 255
Validation binary_cross_entropy = 0.550169
Epoch 256
Validation binary_cross_entropy = 0.543868
Epoch 257
Validation binary_cross_entropy = 0.559732
Epoch 258
Validation binary_cross_entropy = 0.585014
Epoch 259
Loss = 1.0161e+00, PNorm = 62.1835, GNorm = 50.7189, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.605372
Epoch 260
Validation binary_cross_entropy = 0.618787
Epoch 261
Validation binary_cross_entropy = 0.628844
Epoch 262
Validation binary_cross_entropy = 0.631343
Epoch 263
Validation binary_cross_entropy = 0.611540
Epoch 264
Loss = 6.2614e-01, PNorm = 62.1951, GNorm = 39.6443, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.600438
Epoch 265
Validation binary_cross_entropy = 0.593668
Epoch 266
Validation binary_cross_entropy = 0.588503
Epoch 267
Validation binary_cross_entropy = 0.581500
Epoch 268
Validation binary_cross_entropy = 0.586041
Epoch 269
Loss = 6.7283e-01, PNorm = 62.2053, GNorm = 21.8727, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.603085
Epoch 270
Validation binary_cross_entropy = 0.612004
Epoch 271
Validation binary_cross_entropy = 0.616678
Epoch 272
Validation binary_cross_entropy = 0.614918
Epoch 273
Validation binary_cross_entropy = 0.614032
Epoch 274
Loss = 9.6059e-01, PNorm = 62.2231, GNorm = 80.0650, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.613927
Epoch 275
Validation binary_cross_entropy = 0.603490
Epoch 276
Validation binary_cross_entropy = 0.591871
Epoch 277
Validation binary_cross_entropy = 0.589422
Epoch 278
Validation binary_cross_entropy = 0.578038
Epoch 279
Loss = 7.5441e-01, PNorm = 62.2306, GNorm = 44.6160, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.565210
Epoch 280
Validation binary_cross_entropy = 0.536818
Epoch 281
Validation binary_cross_entropy = 0.518316
Epoch 282
Validation binary_cross_entropy = 0.527831
Epoch 283
Validation binary_cross_entropy = 0.549765
Epoch 284
Loss = 1.1050e+00, PNorm = 62.2211, GNorm = 77.6910, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.558061
Epoch 285
Validation binary_cross_entropy = 0.556774
Epoch 286
Validation binary_cross_entropy = 0.559461
Epoch 287
Validation binary_cross_entropy = 0.555927
Epoch 288
Validation binary_cross_entropy = 0.552763
Epoch 289
Loss = 5.6108e-01, PNorm = 62.2313, GNorm = 33.6608, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.551506
Epoch 290
Validation binary_cross_entropy = 0.538920
Epoch 291
Validation binary_cross_entropy = 0.525505
Epoch 292
Validation binary_cross_entropy = 0.508418
Epoch 293
Validation binary_cross_entropy = 0.501558
Epoch 294
Loss = 7.4875e-01, PNorm = 62.2447, GNorm = 41.9114, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.496713
Epoch 295
Validation binary_cross_entropy = 0.495324
Epoch 296
Validation binary_cross_entropy = 0.513176
Epoch 297
Validation binary_cross_entropy = 0.525868
Epoch 298
Validation binary_cross_entropy = 0.543781
Epoch 299
Loss = 6.3326e-01, PNorm = 62.2476, GNorm = 48.4205, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.555233
Model 0 best validation binary_cross_entropy = 0.318769 on epoch 180
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.279857
Ensemble test binary_cross_entropy = 0.279857
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Warning: Found a task with predictions all 0s or all 1s
Validation binary_cross_entropy = nan
Epoch 1
Warning: Found a task with predictions all 0s or all 1s
Validation binary_cross_entropy = nan
Epoch 2
Validation binary_cross_entropy = 7.561675
Epoch 3
Validation binary_cross_entropy = 6.572635
Epoch 4
Loss = 7.9290e+01, PNorm = 60.3409, GNorm = 1203.8663, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 26.008768
Epoch 5
Validation binary_cross_entropy = 27.963102
Epoch 6
Validation binary_cross_entropy = 15.720718
Epoch 7
Validation binary_cross_entropy = 13.183460
Epoch 8
Validation binary_cross_entropy = 24.396246
Epoch 9
Loss = 5.0374e+01, PNorm = 60.4719, GNorm = 1047.2230, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 49.096226
Epoch 10
Validation binary_cross_entropy = 9.080103
Epoch 11
Validation binary_cross_entropy = 7.687725
Epoch 12
Validation binary_cross_entropy = 35.184246
Epoch 13
Validation binary_cross_entropy = 51.985023
Epoch 14
Loss = 7.5910e+01, PNorm = 60.6421, GNorm = 1732.6577, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 8.363810
Epoch 15
Validation binary_cross_entropy = 5.146113
Epoch 16
Validation binary_cross_entropy = 9.403181
Epoch 17
Validation binary_cross_entropy = 20.864975
Epoch 18
Validation binary_cross_entropy = 9.767666
Epoch 19
Loss = 6.5841e+01, PNorm = 60.8075, GNorm = 872.6040, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 5.793309
Epoch 20
Validation binary_cross_entropy = 4.738776
Epoch 21
Validation binary_cross_entropy = 37.377647
Epoch 22
Validation binary_cross_entropy = 4.435432
Epoch 23
Validation binary_cross_entropy = 5.159330
Epoch 24
Loss = 5.2938e+01, PNorm = 60.9358, GNorm = 1023.0557, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 5.321002
Epoch 25
Validation binary_cross_entropy = 15.137049
Epoch 26
Validation binary_cross_entropy = 18.915827
Epoch 27
Validation binary_cross_entropy = 5.753456
Epoch 28
Validation binary_cross_entropy = 3.836977
Epoch 29
Loss = 2.9533e+01, PNorm = 61.0438, GNorm = 548.6277, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 14.290774
Epoch 30
Validation binary_cross_entropy = 35.578312
Epoch 31
Validation binary_cross_entropy = 34.933613
Epoch 32
Validation binary_cross_entropy = 12.574535
Epoch 33
Validation binary_cross_entropy = 4.454776
Epoch 34
Loss = 3.1111e+01, PNorm = 61.1476, GNorm = 685.8825, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.014526
Epoch 35
Validation binary_cross_entropy = 20.427822
Epoch 36
Validation binary_cross_entropy = 16.830914
Epoch 37
Validation binary_cross_entropy = 3.338462
Epoch 38
Validation binary_cross_entropy = 2.755267
Epoch 39
Loss = 2.8914e+01, PNorm = 61.2342, GNorm = 641.5897, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.589437
Epoch 40
Validation binary_cross_entropy = 5.040226
Epoch 41
Validation binary_cross_entropy = 3.404932
Epoch 42
Validation binary_cross_entropy = 3.499189
Epoch 43
Validation binary_cross_entropy = 10.259268
Epoch 44
Loss = 2.7247e+01, PNorm = 61.2781, GNorm = 364.6451, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 11.732160
Epoch 45
Validation binary_cross_entropy = 7.406443
Epoch 46
Validation binary_cross_entropy = 4.191894
Epoch 47
Validation binary_cross_entropy = 6.739165
Epoch 48
Validation binary_cross_entropy = 8.772866
Epoch 49
Loss = 1.1732e+01, PNorm = 61.3363, GNorm = 525.9883, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 10.699782
Epoch 50
Validation binary_cross_entropy = 10.695989
Epoch 51
Validation binary_cross_entropy = 5.731980
Epoch 52
Validation binary_cross_entropy = 4.982634
Epoch 53
Validation binary_cross_entropy = 8.228946
Epoch 54
Loss = 1.2252e+01, PNorm = 61.3758, GNorm = 473.6550, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 13.067977
Epoch 55
Validation binary_cross_entropy = 14.844918
Epoch 56
Validation binary_cross_entropy = 17.933094
Epoch 57
Validation binary_cross_entropy = 17.875099
Epoch 58
Validation binary_cross_entropy = 17.655874
Epoch 59
Loss = 9.6430e+00, PNorm = 61.4021, GNorm = 326.3261, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 17.966526
Epoch 60
Validation binary_cross_entropy = 12.462189
Epoch 61
Validation binary_cross_entropy = 5.891666
Epoch 62
Validation binary_cross_entropy = 6.285143
Epoch 63
Validation binary_cross_entropy = 7.169137
Epoch 64
Loss = 1.7038e+01, PNorm = 61.4387, GNorm = 391.6441, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 10.688981
Epoch 65
Validation binary_cross_entropy = 7.761263
Epoch 66
Validation binary_cross_entropy = 5.440536
Epoch 67
Validation binary_cross_entropy = 3.365900
Epoch 68
Validation binary_cross_entropy = 2.971571
Epoch 69
Loss = 1.1098e+01, PNorm = 61.4627, GNorm = 330.5658, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 4.458823
Epoch 70
Validation binary_cross_entropy = 4.582261
Epoch 71
Validation binary_cross_entropy = 3.532166
Epoch 72
Validation binary_cross_entropy = 5.060470
Epoch 73
Validation binary_cross_entropy = 3.635667
Epoch 74
Loss = 1.2068e+01, PNorm = 61.4877, GNorm = 405.2707, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.635442
Epoch 75
Validation binary_cross_entropy = 1.949837
Epoch 76
Validation binary_cross_entropy = 1.646803
Epoch 77
Validation binary_cross_entropy = 1.666095
Epoch 78
Validation binary_cross_entropy = 2.943459
Epoch 79
Loss = 8.6248e+00, PNorm = 61.5294, GNorm = 212.8197, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.273296
Epoch 80
Validation binary_cross_entropy = 3.206710
Epoch 81
Validation binary_cross_entropy = 3.615966
Epoch 82
Validation binary_cross_entropy = 3.256186
Epoch 83
Validation binary_cross_entropy = 2.593939
Epoch 84
Loss = 9.0321e+00, PNorm = 61.5608, GNorm = 285.1487, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.129155
Epoch 85
Validation binary_cross_entropy = 1.189975
Epoch 86
Validation binary_cross_entropy = 1.271618
Epoch 87
Validation binary_cross_entropy = 1.162242
Epoch 88
Validation binary_cross_entropy = 2.544180
Epoch 89
Loss = 8.7210e+00, PNorm = 61.5730, GNorm = 285.2640, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 6.074310
Epoch 90
Validation binary_cross_entropy = 5.422152
Epoch 91
Validation binary_cross_entropy = 1.313685
Epoch 92
Validation binary_cross_entropy = 1.160339
Epoch 93
Validation binary_cross_entropy = 1.163278
Epoch 94
Loss = 7.2176e+00, PNorm = 61.6088, GNorm = 227.6530, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.060765
Epoch 95
Validation binary_cross_entropy = 0.974982
Epoch 96
Validation binary_cross_entropy = 1.241218
Epoch 97
Validation binary_cross_entropy = 3.555673
Epoch 98
Validation binary_cross_entropy = 2.743235
Epoch 99
Loss = 7.7465e+00, PNorm = 61.6606, GNorm = 246.9909, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.917108
Epoch 100
Validation binary_cross_entropy = 1.940916
Epoch 101
Validation binary_cross_entropy = 2.741463
Epoch 102
Validation binary_cross_entropy = 4.008388
Epoch 103
Validation binary_cross_entropy = 3.904424
Epoch 104
Loss = 1.0389e+01, PNorm = 61.6938, GNorm = 295.0719, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 2.085881
Epoch 105
Validation binary_cross_entropy = 1.765145
Epoch 106
Validation binary_cross_entropy = 0.821306
Epoch 107
Validation binary_cross_entropy = 0.766012
Epoch 108
Validation binary_cross_entropy = 1.052823
Epoch 109
Loss = 5.5742e+00, PNorm = 61.7290, GNorm = 179.7338, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 3.329703
Epoch 110
Validation binary_cross_entropy = 4.817275
Epoch 111
Validation binary_cross_entropy = 8.541087
Epoch 112
Validation binary_cross_entropy = 6.070592
Epoch 113
Validation binary_cross_entropy = 1.835840
Epoch 114
Loss = 5.6139e+00, PNorm = 61.7450, GNorm = 180.6041, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.589009
Epoch 115
Validation binary_cross_entropy = 0.501985
Epoch 116
Validation binary_cross_entropy = 0.504074
Epoch 117
Validation binary_cross_entropy = 0.778509
Epoch 118
Validation binary_cross_entropy = 1.146396
Epoch 119
Loss = 3.3751e+00, PNorm = 61.7528, GNorm = 205.8962, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.481367
Epoch 120
Validation binary_cross_entropy = 1.330229
Epoch 121
Validation binary_cross_entropy = 0.735725
Epoch 122
Validation binary_cross_entropy = 0.522540
Epoch 123
Validation binary_cross_entropy = 0.498407
Epoch 124
Loss = 4.9799e+00, PNorm = 61.7786, GNorm = 158.5543, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.492799
Epoch 125
Validation binary_cross_entropy = 0.528859
Epoch 126
Validation binary_cross_entropy = 0.715772
Epoch 127
Validation binary_cross_entropy = 1.141309
Epoch 128
Validation binary_cross_entropy = 1.743112
Epoch 129
Loss = 4.0936e+00, PNorm = 61.8072, GNorm = 166.6291, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.741246
Epoch 130
Validation binary_cross_entropy = 1.126827
Epoch 131
Validation binary_cross_entropy = 0.605103
Epoch 132
Validation binary_cross_entropy = 0.471112
Epoch 133
Validation binary_cross_entropy = 0.463050
Epoch 134
Loss = 3.6162e+00, PNorm = 61.8384, GNorm = 171.2478, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.457914
Epoch 135
Validation binary_cross_entropy = 0.470884
Epoch 136
Validation binary_cross_entropy = 0.487285
Epoch 137
Validation binary_cross_entropy = 0.521363
Epoch 138
Validation binary_cross_entropy = 0.619928
Epoch 139
Loss = 2.3027e+00, PNorm = 61.8601, GNorm = 92.5058, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.763437
Epoch 140
Validation binary_cross_entropy = 0.856285
Epoch 141
Validation binary_cross_entropy = 0.968105
Epoch 142
Validation binary_cross_entropy = 0.902114
Epoch 143
Validation binary_cross_entropy = 0.673979
Epoch 144
Loss = 3.0512e+00, PNorm = 61.8707, GNorm = 134.0736, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.470729
Epoch 145
Validation binary_cross_entropy = 0.392985
Epoch 146
Validation binary_cross_entropy = 0.386908
Epoch 147
Validation binary_cross_entropy = 0.392062
Epoch 148
Validation binary_cross_entropy = 0.381593
Epoch 149
Loss = 2.9278e+00, PNorm = 61.8786, GNorm = 174.2332, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.369907
Epoch 150
Validation binary_cross_entropy = 0.364022
Epoch 151
Validation binary_cross_entropy = 0.353430
Epoch 152
Validation binary_cross_entropy = 0.359575
Epoch 153
Validation binary_cross_entropy = 0.369657
Epoch 154
Loss = 2.1431e+00, PNorm = 61.8991, GNorm = 92.2584, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.338404
Epoch 155
Validation binary_cross_entropy = 0.327257
Epoch 156
Validation binary_cross_entropy = 0.322941
Epoch 157
Validation binary_cross_entropy = 0.324331
Epoch 158
Validation binary_cross_entropy = 0.325794
Epoch 159
Loss = 2.9651e+00, PNorm = 61.9248, GNorm = 204.2193, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.332425
Epoch 160
Validation binary_cross_entropy = 0.357583
Epoch 161
Validation binary_cross_entropy = 0.390677
Epoch 162
Validation binary_cross_entropy = 0.417028
Epoch 163
Validation binary_cross_entropy = 0.439063
Epoch 164
Loss = 3.0955e+00, PNorm = 61.9460, GNorm = 86.2598, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.444272
Epoch 165
Validation binary_cross_entropy = 0.425374
Epoch 166
Validation binary_cross_entropy = 0.410859
Epoch 167
Validation binary_cross_entropy = 0.391761
Epoch 168
Validation binary_cross_entropy = 0.379333
Epoch 169
Loss = 3.2259e+00, PNorm = 61.9662, GNorm = 164.6896, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.373277
Epoch 170
Validation binary_cross_entropy = 0.396485
Epoch 171
Validation binary_cross_entropy = 0.455291
Epoch 172
Validation binary_cross_entropy = 0.533555
Epoch 173
Validation binary_cross_entropy = 0.582416
Epoch 174
Loss = 3.3777e+00, PNorm = 61.9742, GNorm = 129.7762, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.577321
Epoch 175
Validation binary_cross_entropy = 0.533559
Epoch 176
Validation binary_cross_entropy = 0.457280
Epoch 177
Validation binary_cross_entropy = 0.425864
Epoch 178
Validation binary_cross_entropy = 0.453130
Epoch 179
Loss = 2.9543e+00, PNorm = 61.9940, GNorm = 122.6871, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.480516
Epoch 180
Validation binary_cross_entropy = 0.486026
Epoch 181
Validation binary_cross_entropy = 0.458213
Epoch 182
Validation binary_cross_entropy = 0.404650
Epoch 183
Validation binary_cross_entropy = 0.366408
Epoch 184
Loss = 2.5282e+00, PNorm = 62.0251, GNorm = 143.2461, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.358291
Epoch 185
Validation binary_cross_entropy = 0.381270
Epoch 186
Validation binary_cross_entropy = 0.425200
Epoch 187
Validation binary_cross_entropy = 0.457173
Epoch 188
Validation binary_cross_entropy = 0.457298
Epoch 189
Loss = 3.1720e+00, PNorm = 62.0438, GNorm = 111.0548, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.458745
Epoch 190
Validation binary_cross_entropy = 0.449083
Epoch 191
Validation binary_cross_entropy = 0.400831
Epoch 192
Validation binary_cross_entropy = 0.371154
Epoch 193
Validation binary_cross_entropy = 0.352915
Epoch 194
Loss = 1.3673e+00, PNorm = 62.0723, GNorm = 86.4619, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.343032
Epoch 195
Validation binary_cross_entropy = 0.346227
Epoch 196
Validation binary_cross_entropy = 0.364267
Epoch 197
Validation binary_cross_entropy = 0.415493
Epoch 198
Validation binary_cross_entropy = 0.465626
Epoch 199
Loss = 2.2507e+00, PNorm = 62.0920, GNorm = 118.9061, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.535004
Epoch 200
Validation binary_cross_entropy = 0.617058
Epoch 201
Validation binary_cross_entropy = 0.626117
Epoch 202
Validation binary_cross_entropy = 0.512714
Epoch 203
Validation binary_cross_entropy = 0.468386
Epoch 204
Loss = 1.8283e+00, PNorm = 62.1294, GNorm = 108.2650, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.419333
Epoch 205
Validation binary_cross_entropy = 0.402259
Epoch 206
Validation binary_cross_entropy = 0.398981
Epoch 207
Validation binary_cross_entropy = 0.412678
Epoch 208
Validation binary_cross_entropy = 0.426066
Epoch 209
Loss = 1.1175e+00, PNorm = 62.1621, GNorm = 77.7568, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.432844
Epoch 210
Validation binary_cross_entropy = 0.443271
Epoch 211
Validation binary_cross_entropy = 0.467333
Epoch 212
Validation binary_cross_entropy = 0.492599
Epoch 213
Validation binary_cross_entropy = 0.505181
Epoch 214
Loss = 1.5775e+00, PNorm = 62.1782, GNorm = 87.4031, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.502080
Epoch 215
Validation binary_cross_entropy = 0.479356
Epoch 216
Validation binary_cross_entropy = 0.459922
Epoch 217
Validation binary_cross_entropy = 0.443171
Epoch 218
Validation binary_cross_entropy = 0.424491
Epoch 219
Loss = 1.5248e+00, PNorm = 62.1867, GNorm = 69.7184, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.422127
Epoch 220
Validation binary_cross_entropy = 0.446425
Epoch 221
Validation binary_cross_entropy = 0.474899
Epoch 222
Validation binary_cross_entropy = 0.484381
Epoch 223
Validation binary_cross_entropy = 0.484428
Epoch 224
Loss = 2.6468e+00, PNorm = 62.1984, GNorm = 108.4154, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.468547
Epoch 225
Validation binary_cross_entropy = 0.461686
Epoch 226
Validation binary_cross_entropy = 0.476540
Epoch 227
Validation binary_cross_entropy = 0.484679
Epoch 228
Validation binary_cross_entropy = 0.495247
Epoch 229
Loss = 1.2115e+00, PNorm = 62.2167, GNorm = 69.3670, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.493028
Epoch 230
Validation binary_cross_entropy = 0.489129
Epoch 231
Validation binary_cross_entropy = 0.479565
Epoch 232
Validation binary_cross_entropy = 0.490860
Epoch 233
Validation binary_cross_entropy = 0.495827
Epoch 234
Loss = 1.3237e+00, PNorm = 62.2347, GNorm = 85.8606, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.507011
Epoch 235
Validation binary_cross_entropy = 0.528981
Epoch 236
Validation binary_cross_entropy = 0.540671
Epoch 237
Validation binary_cross_entropy = 0.543236
Epoch 238
Validation binary_cross_entropy = 0.522349
Epoch 239
Loss = 1.3329e+00, PNorm = 62.2593, GNorm = 48.6712, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.527017
Epoch 240
Validation binary_cross_entropy = 0.497764
Epoch 241
Validation binary_cross_entropy = 0.449921
Epoch 242
Validation binary_cross_entropy = 0.405175
Epoch 243
Validation binary_cross_entropy = 0.397011
Epoch 244
Loss = 1.5820e+00, PNorm = 62.2827, GNorm = 50.3368, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.416586
Epoch 245
Validation binary_cross_entropy = 0.436460
Epoch 246
Validation binary_cross_entropy = 0.428711
Epoch 247
Validation binary_cross_entropy = 0.423706
Epoch 248
Validation binary_cross_entropy = 0.407641
Epoch 249
Loss = 8.7236e-01, PNorm = 62.3028, GNorm = 79.4859, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.394204
Epoch 250
Validation binary_cross_entropy = 0.398873
Epoch 251
Validation binary_cross_entropy = 0.420832
Epoch 252
Validation binary_cross_entropy = 0.474260
Epoch 253
Validation binary_cross_entropy = 0.516074
Epoch 254
Loss = 1.5153e+00, PNorm = 62.3075, GNorm = 89.6624, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.547909
Epoch 255
Validation binary_cross_entropy = 0.550471
Epoch 256
Validation binary_cross_entropy = 0.535682
Epoch 257
Validation binary_cross_entropy = 0.509522
Epoch 258
Validation binary_cross_entropy = 0.470290
Epoch 259
Loss = 9.5675e-01, PNorm = 62.3163, GNorm = 64.1569, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.439962
Epoch 260
Validation binary_cross_entropy = 0.422816
Epoch 261
Validation binary_cross_entropy = 0.427502
Epoch 262
Validation binary_cross_entropy = 0.448938
Epoch 263
Validation binary_cross_entropy = 0.480815
Epoch 264
Loss = 8.2828e-01, PNorm = 62.3139, GNorm = 38.9766, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.508233
Epoch 265
Validation binary_cross_entropy = 0.531410
Epoch 266
Validation binary_cross_entropy = 0.538934
Epoch 267
Validation binary_cross_entropy = 0.542315
Epoch 268
Validation binary_cross_entropy = 0.546608
Epoch 269
Loss = 1.2873e+00, PNorm = 62.3217, GNorm = 68.5703, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.536530
Epoch 270
Validation binary_cross_entropy = 0.531564
Epoch 271
Validation binary_cross_entropy = 0.526078
Epoch 272
Validation binary_cross_entropy = 0.520509
Epoch 273
Validation binary_cross_entropy = 0.518528
Epoch 274
Loss = 1.2014e+00, PNorm = 62.3401, GNorm = 54.8945, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.521617
Epoch 275
Validation binary_cross_entropy = 0.515513
Epoch 276
Validation binary_cross_entropy = 0.508575
Epoch 277
Validation binary_cross_entropy = 0.522195
Epoch 278
Validation binary_cross_entropy = 0.535025
Epoch 279
Loss = 8.1517e-01, PNorm = 62.3599, GNorm = 50.1495, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.538131
Epoch 280
Validation binary_cross_entropy = 0.529419
Epoch 281
Validation binary_cross_entropy = 0.520567
Epoch 282
Validation binary_cross_entropy = 0.515832
Epoch 283
Validation binary_cross_entropy = 0.519497
Epoch 284
Loss = 8.3093e-01, PNorm = 62.3873, GNorm = 40.1125, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.527708
Epoch 285
Validation binary_cross_entropy = 0.531412
Epoch 286
Validation binary_cross_entropy = 0.535770
Epoch 287
Validation binary_cross_entropy = 0.544034
Epoch 288
Validation binary_cross_entropy = 0.546397
Epoch 289
Loss = 9.6549e-01, PNorm = 62.4087, GNorm = 73.3662, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.548710
Epoch 290
Validation binary_cross_entropy = 0.549408
Epoch 291
Validation binary_cross_entropy = 0.543446
Epoch 292
Validation binary_cross_entropy = 0.540233
Epoch 293
Validation binary_cross_entropy = 0.530913
Epoch 294
Loss = 1.0203e+00, PNorm = 62.4284, GNorm = 65.3204, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.504013
Epoch 295
Validation binary_cross_entropy = 0.490033
Epoch 296
Validation binary_cross_entropy = 0.474879
Epoch 297
Validation binary_cross_entropy = 0.473240
Epoch 298
Validation binary_cross_entropy = 0.480716
Epoch 299
Loss = 5.7098e-01, PNorm = 62.4430, GNorm = 30.5512, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.489314
Model 0 best validation binary_cross_entropy = 0.322941 on epoch 156
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.303652
Ensemble test binary_cross_entropy = 0.303652
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.906044
Epoch 1
Validation binary_cross_entropy = 87.582840
Epoch 2
Validation binary_cross_entropy = 4.962898
Epoch 3
Validation binary_cross_entropy = 17.780090
Epoch 4
Loss = 5.6243e+01, PNorm = 60.3413, GNorm = 1159.0343, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 26.967009
Epoch 5
Validation binary_cross_entropy = 18.035334
Epoch 6
Validation binary_cross_entropy = 8.982472
Epoch 7
Validation binary_cross_entropy = 22.814672
Epoch 8
Validation binary_cross_entropy = 59.599430
Epoch 9
Loss = 9.7469e+01, PNorm = 60.4609, GNorm = 2265.1239, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 43.498665
Epoch 10
Validation binary_cross_entropy = 8.943143
Epoch 11
Validation binary_cross_entropy = 8.800366
Epoch 12
Validation binary_cross_entropy = 14.755364
Epoch 13
Validation binary_cross_entropy = 38.126404
Epoch 14
Loss = 6.7598e+01, PNorm = 60.6140, GNorm = 1149.8985, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 21.681072
Epoch 15
Validation binary_cross_entropy = 7.590057
Epoch 16
Validation binary_cross_entropy = 14.570790
Epoch 17
Validation binary_cross_entropy = 24.417166
Epoch 18
Validation binary_cross_entropy = 15.545925
Epoch 19
Loss = 5.2819e+01, PNorm = 60.7489, GNorm = 853.5028, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 6.830088
Epoch 20
Validation binary_cross_entropy = 9.802164
Epoch 21
Validation binary_cross_entropy = 23.081095
Epoch 22
Validation binary_cross_entropy = 10.895691
Epoch 23
Validation binary_cross_entropy = 6.417763
Epoch 24
Loss = 2.3029e+01, PNorm = 60.8376, GNorm = 568.4860, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.450795
Epoch 25
Validation binary_cross_entropy = 18.445976
Epoch 26
Validation binary_cross_entropy = 42.102814
Epoch 27
Validation binary_cross_entropy = 17.503681
Epoch 28
Validation binary_cross_entropy = 6.961417
Epoch 29
Loss = 3.1134e+01, PNorm = 60.8956, GNorm = 607.4461, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 7.119428
Epoch 30
Validation binary_cross_entropy = 7.197264
Epoch 31
Validation binary_cross_entropy = 11.834146
Epoch 32
Validation binary_cross_entropy = 10.311285
Epoch 33
Validation binary_cross_entropy = 5.836973
Epoch 34
Loss = 3.0101e+01, PNorm = 60.9432, GNorm = 666.1244, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.140221
Epoch 35
Validation binary_cross_entropy = 5.365102
Epoch 36
Validation binary_cross_entropy = 6.674145
Epoch 37
Validation binary_cross_entropy = 8.583170
Epoch 38
Validation binary_cross_entropy = 8.128968
Epoch 39
Loss = 3.8261e+01, PNorm = 60.9865, GNorm = 810.8097, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 10.451546
Epoch 40
Validation binary_cross_entropy = 11.387874
Epoch 41
Validation binary_cross_entropy = 11.886070
Epoch 42
Validation binary_cross_entropy = 9.796338
Epoch 43
Validation binary_cross_entropy = 7.181547
Epoch 44
Loss = 2.0341e+01, PNorm = 61.0303, GNorm = 442.2976, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 5.095289
Epoch 45
Validation binary_cross_entropy = 4.657763
Epoch 46
Validation binary_cross_entropy = 4.319635
Epoch 47
Validation binary_cross_entropy = 4.622468
Epoch 48
Validation binary_cross_entropy = 2.768131
Epoch 49
Loss = 1.7372e+01, PNorm = 61.1081, GNorm = 618.4877, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 2.778625
Epoch 50
Validation binary_cross_entropy = 3.397265
Epoch 51
Validation binary_cross_entropy = 3.470717
Epoch 52
Validation binary_cross_entropy = 3.865365
Epoch 53
Validation binary_cross_entropy = 11.047421
Epoch 54
Loss = 1.9671e+01, PNorm = 61.1726, GNorm = 639.7041, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 19.401649
Epoch 55
Validation binary_cross_entropy = 6.311973
Epoch 56
Validation binary_cross_entropy = 3.462238
Epoch 57
Validation binary_cross_entropy = 3.409747
Epoch 58
Validation binary_cross_entropy = 8.399544
Epoch 59
Loss = 1.3520e+01, PNorm = 61.2892, GNorm = 301.3483, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 14.308685
Epoch 60
Validation binary_cross_entropy = 11.357692
Epoch 61
Validation binary_cross_entropy = 11.008825
Epoch 62
Validation binary_cross_entropy = 9.560891
Epoch 63
Validation binary_cross_entropy = 7.176674
Epoch 64
Loss = 9.1518e+00, PNorm = 61.3479, GNorm = 642.8039, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 6.646179
Epoch 65
Validation binary_cross_entropy = 6.115425
Epoch 66
Validation binary_cross_entropy = 6.002309
Epoch 67
Validation binary_cross_entropy = 9.133135
Epoch 68
Validation binary_cross_entropy = 13.567347
Epoch 69
Loss = 1.0965e+01, PNorm = 61.4093, GNorm = 196.1828, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 10.699280
Epoch 70
Validation binary_cross_entropy = 4.947308
Epoch 71
Validation binary_cross_entropy = 1.818110
Epoch 72
Validation binary_cross_entropy = 1.701918
Epoch 73
Validation binary_cross_entropy = 1.846999
Epoch 74
Loss = 1.1627e+01, PNorm = 61.4498, GNorm = 377.9130, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.493932
Epoch 75
Validation binary_cross_entropy = 11.880071
Epoch 76
Validation binary_cross_entropy = 18.283741
Epoch 77
Validation binary_cross_entropy = 9.101548
Epoch 78
Validation binary_cross_entropy = 3.159108
Epoch 79
Loss = 8.5355e+00, PNorm = 61.4829, GNorm = 233.4896, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.582741
Epoch 80
Validation binary_cross_entropy = 1.582788
Epoch 81
Validation binary_cross_entropy = 2.184312
Epoch 82
Validation binary_cross_entropy = 4.131869
Epoch 83
Validation binary_cross_entropy = 6.410430
Epoch 84
Loss = 9.5341e+00, PNorm = 61.5031, GNorm = 281.4530, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 9.568777
Epoch 85
Validation binary_cross_entropy = 8.035388
Epoch 86
Validation binary_cross_entropy = 6.272020
Epoch 87
Validation binary_cross_entropy = 4.577754
Epoch 88
Validation binary_cross_entropy = 3.769136
Epoch 89
Loss = 8.1920e+00, PNorm = 61.5252, GNorm = 209.3941, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 7.538347
Epoch 90
Validation binary_cross_entropy = 9.785284
Epoch 91
Validation binary_cross_entropy = 10.742826
Epoch 92
Validation binary_cross_entropy = 10.478117
Epoch 93
Validation binary_cross_entropy = 7.633672
Epoch 94
Loss = 1.1517e+01, PNorm = 61.5454, GNorm = 438.7511, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 5.764889
Epoch 95
Validation binary_cross_entropy = 2.888272
Epoch 96
Validation binary_cross_entropy = 1.830471
Epoch 97
Validation binary_cross_entropy = 1.662569
Epoch 98
Validation binary_cross_entropy = 1.695170
Epoch 99
Loss = 9.5998e+00, PNorm = 61.5662, GNorm = 313.5805, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.633368
Epoch 100
Validation binary_cross_entropy = 1.587807
Epoch 101
Validation binary_cross_entropy = 1.684752
Epoch 102
Validation binary_cross_entropy = 2.074831
Epoch 103
Validation binary_cross_entropy = 3.348543
Epoch 104
Loss = 5.1754e+00, PNorm = 61.5905, GNorm = 275.5495, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 3.581629
Epoch 105
Validation binary_cross_entropy = 2.412568
Epoch 106
Validation binary_cross_entropy = 1.762981
Epoch 107
Validation binary_cross_entropy = 0.803565
Epoch 108
Validation binary_cross_entropy = 0.667990
Epoch 109
Loss = 5.4430e+00, PNorm = 61.6171, GNorm = 232.6046, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.780997
Epoch 110
Validation binary_cross_entropy = 0.972105
Epoch 111
Validation binary_cross_entropy = 1.264962
Epoch 112
Validation binary_cross_entropy = 1.774968
Epoch 113
Validation binary_cross_entropy = 1.569599
Epoch 114
Loss = 4.7084e+00, PNorm = 61.6650, GNorm = 167.7420, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.649733
Epoch 115
Validation binary_cross_entropy = 1.525442
Epoch 116
Validation binary_cross_entropy = 1.480339
Epoch 117
Validation binary_cross_entropy = 0.803093
Epoch 118
Validation binary_cross_entropy = 0.753078
Epoch 119
Loss = 3.1052e+00, PNorm = 61.7004, GNorm = 159.9525, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.638443
Epoch 120
Validation binary_cross_entropy = 0.491995
Epoch 121
Validation binary_cross_entropy = 0.436046
Epoch 122
Validation binary_cross_entropy = 0.461340
Epoch 123
Validation binary_cross_entropy = 0.617039
Epoch 124
Loss = 4.5612e+00, PNorm = 61.7256, GNorm = 211.7794, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.907139
Epoch 125
Validation binary_cross_entropy = 1.246589
Epoch 126
Validation binary_cross_entropy = 1.403808
Epoch 127
Validation binary_cross_entropy = 0.954122
Epoch 128
Validation binary_cross_entropy = 0.511070
Epoch 129
Loss = 4.5080e+00, PNorm = 61.7704, GNorm = 202.3316, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.471927
Epoch 130
Validation binary_cross_entropy = 0.464160
Epoch 131
Validation binary_cross_entropy = 0.486332
Epoch 132
Validation binary_cross_entropy = 0.521494
Epoch 133
Validation binary_cross_entropy = 0.620624
Epoch 134
Loss = 4.9162e+00, PNorm = 61.8123, GNorm = 237.0459, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.647250
Epoch 135
Validation binary_cross_entropy = 0.532163
Epoch 136
Validation binary_cross_entropy = 0.395545
Epoch 137
Validation binary_cross_entropy = 0.359897
Epoch 138
Validation binary_cross_entropy = 0.361608
Epoch 139
Loss = 6.0137e+00, PNorm = 61.8416, GNorm = 135.3522, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.372935
Epoch 140
Validation binary_cross_entropy = 0.374057
Epoch 141
Validation binary_cross_entropy = 0.372374
Epoch 142
Validation binary_cross_entropy = 0.377881
Epoch 143
Validation binary_cross_entropy = 0.350116
Epoch 144
Loss = 3.0419e+00, PNorm = 61.8577, GNorm = 99.0956, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.338368
Epoch 145
Validation binary_cross_entropy = 0.330672
Epoch 146
Validation binary_cross_entropy = 0.326166
Epoch 147
Validation binary_cross_entropy = 0.340988
Epoch 148
Validation binary_cross_entropy = 0.353997
Epoch 149
Loss = 4.3024e+00, PNorm = 61.8819, GNorm = 208.3611, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.340225
Epoch 150
Validation binary_cross_entropy = 0.314470
Epoch 151
Validation binary_cross_entropy = 0.351230
Epoch 152
Validation binary_cross_entropy = 0.340846
Epoch 153
Validation binary_cross_entropy = 0.295212
Epoch 154
Loss = 2.4839e+00, PNorm = 61.8972, GNorm = 150.6877, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.328028
Epoch 155
Validation binary_cross_entropy = 0.447614
Epoch 156
Validation binary_cross_entropy = 0.575654
Epoch 157
Validation binary_cross_entropy = 0.678867
Epoch 158
Validation binary_cross_entropy = 0.627693
Epoch 159
Loss = 1.9876e+00, PNorm = 61.9029, GNorm = 96.9070, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.509561
Epoch 160
Validation binary_cross_entropy = 0.417543
Epoch 161
Validation binary_cross_entropy = 0.347482
Epoch 162
Validation binary_cross_entropy = 0.337111
Epoch 163
Validation binary_cross_entropy = 0.358087
Epoch 164
Loss = 2.0655e+00, PNorm = 61.9152, GNorm = 55.2662, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.387245
Epoch 165
Validation binary_cross_entropy = 0.459193
Epoch 166
Validation binary_cross_entropy = 0.561020
Epoch 167
Validation binary_cross_entropy = 0.637182
Epoch 168
Validation binary_cross_entropy = 0.612377
Epoch 169
Loss = 2.2983e+00, PNorm = 61.9224, GNorm = 105.1661, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.504860
Epoch 170
Validation binary_cross_entropy = 0.444911
Epoch 171
Validation binary_cross_entropy = 0.398741
Epoch 172
Validation binary_cross_entropy = 0.368533
Epoch 173
Validation binary_cross_entropy = 0.358985
Epoch 174
Loss = 3.3206e+00, PNorm = 61.9302, GNorm = 132.2599, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.367901
Epoch 175
Validation binary_cross_entropy = 0.381474
Epoch 176
Validation binary_cross_entropy = 0.425924
Epoch 177
Validation binary_cross_entropy = 0.509380
Epoch 178
Validation binary_cross_entropy = 0.621946
Epoch 179
Loss = 1.3856e+00, PNorm = 61.9434, GNorm = 97.7319, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.715051
Epoch 180
Validation binary_cross_entropy = 0.737505
Epoch 181
Validation binary_cross_entropy = 0.698147
Epoch 182
Validation binary_cross_entropy = 0.609607
Epoch 183
Validation binary_cross_entropy = 0.553079
Epoch 184
Loss = 1.3593e+00, PNorm = 61.9612, GNorm = 43.0641, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.489058
Epoch 185
Validation binary_cross_entropy = 0.437715
Epoch 186
Validation binary_cross_entropy = 0.406800
Epoch 187
Validation binary_cross_entropy = 0.395731
Epoch 188
Validation binary_cross_entropy = 0.408856
Epoch 189
Loss = 2.3292e+00, PNorm = 61.9839, GNorm = 155.2704, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.428277
Epoch 190
Validation binary_cross_entropy = 0.485437
Epoch 191
Validation binary_cross_entropy = 0.564139
Epoch 192
Validation binary_cross_entropy = 0.683092
Epoch 193
Validation binary_cross_entropy = 0.787585
Epoch 194
Loss = 3.1892e+00, PNorm = 61.9865, GNorm = 147.7890, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.880255
Epoch 195
Validation binary_cross_entropy = 0.923745
Epoch 196
Validation binary_cross_entropy = 0.896372
Epoch 197
Validation binary_cross_entropy = 0.845263
Epoch 198
Validation binary_cross_entropy = 0.752042
Epoch 199
Loss = 1.9888e+00, PNorm = 61.9948, GNorm = 157.2995, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.666754
Epoch 200
Validation binary_cross_entropy = 0.599069
Epoch 201
Validation binary_cross_entropy = 0.550317
Epoch 202
Validation binary_cross_entropy = 0.538770
Epoch 203
Validation binary_cross_entropy = 0.546587
Epoch 204
Loss = 7.0707e-01, PNorm = 62.0077, GNorm = 87.9334, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.568953
Epoch 205
Validation binary_cross_entropy = 0.589994
Epoch 206
Validation binary_cross_entropy = 0.617113
Epoch 207
Validation binary_cross_entropy = 0.660819
Epoch 208
Validation binary_cross_entropy = 0.662348
Epoch 209
Loss = 1.4985e+00, PNorm = 62.0099, GNorm = 96.9472, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.673586
Epoch 210
Validation binary_cross_entropy = 0.683132
Epoch 211
Validation binary_cross_entropy = 0.698674
Epoch 212
Validation binary_cross_entropy = 0.690017
Epoch 213
Validation binary_cross_entropy = 0.684848
Epoch 214
Loss = 1.2525e+00, PNorm = 62.0056, GNorm = 84.8269, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.689463
Epoch 215
Validation binary_cross_entropy = 0.681199
Epoch 216
Validation binary_cross_entropy = 0.664706
Epoch 217
Validation binary_cross_entropy = 0.639262
Epoch 218
Validation binary_cross_entropy = 0.626637
Epoch 219
Loss = 1.8140e+00, PNorm = 62.0086, GNorm = 124.7730, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.624954
Epoch 220
Validation binary_cross_entropy = 0.618142
Epoch 221
Validation binary_cross_entropy = 0.625008
Epoch 222
Validation binary_cross_entropy = 0.638085
Epoch 223
Validation binary_cross_entropy = 0.633830
Epoch 224
Loss = 1.3868e+00, PNorm = 62.0190, GNorm = 69.1295, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.618092
Epoch 225
Validation binary_cross_entropy = 0.602199
Epoch 226
Validation binary_cross_entropy = 0.590885
Epoch 227
Validation binary_cross_entropy = 0.582596
Epoch 228
Validation binary_cross_entropy = 0.581115
Epoch 229
Loss = 9.7382e-01, PNorm = 62.0352, GNorm = 86.2368, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.573624
Epoch 230
Validation binary_cross_entropy = 0.575820
Epoch 231
Validation binary_cross_entropy = 0.577148
Epoch 232
Validation binary_cross_entropy = 0.596715
Epoch 233
Validation binary_cross_entropy = 0.624423
Epoch 234
Loss = 1.3261e+00, PNorm = 62.0524, GNorm = 69.4079, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.634505
Epoch 235
Validation binary_cross_entropy = 0.625051
Epoch 236
Validation binary_cross_entropy = 0.602382
Epoch 237
Validation binary_cross_entropy = 0.592170
Epoch 238
Validation binary_cross_entropy = 0.570460
Epoch 239
Loss = 1.3039e+00, PNorm = 62.0605, GNorm = 70.6054, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.543150
Epoch 240
Validation binary_cross_entropy = 0.522599
Epoch 241
Validation binary_cross_entropy = 0.534870
Epoch 242
Validation binary_cross_entropy = 0.549048
Epoch 243
Validation binary_cross_entropy = 0.569593
Epoch 244
Loss = 1.0537e+00, PNorm = 62.0644, GNorm = 49.3661, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.580352
Epoch 245
Validation binary_cross_entropy = 0.593361
Epoch 246
Validation binary_cross_entropy = 0.599719
Epoch 247
Validation binary_cross_entropy = 0.603693
Epoch 248
Validation binary_cross_entropy = 0.600887
Epoch 249
Loss = 7.8202e-01, PNorm = 62.0737, GNorm = 94.1418, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.597153
Epoch 250
Validation binary_cross_entropy = 0.603492
Epoch 251
Validation binary_cross_entropy = 0.591834
Epoch 252
Validation binary_cross_entropy = 0.589957
Epoch 253
Validation binary_cross_entropy = 0.598969
Epoch 254
Loss = 1.2798e+00, PNorm = 62.0811, GNorm = 64.2569, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.618567
Epoch 255
Validation binary_cross_entropy = 0.643947
Epoch 256
Validation binary_cross_entropy = 0.660573
Epoch 257
Validation binary_cross_entropy = 0.651009
Epoch 258
Validation binary_cross_entropy = 0.631995
Epoch 259
Loss = 8.4840e-01, PNorm = 62.0923, GNorm = 48.7318, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.613830
Epoch 260
Validation binary_cross_entropy = 0.604101
Epoch 261
Validation binary_cross_entropy = 0.599244
Epoch 262
Validation binary_cross_entropy = 0.595457
Epoch 263
Validation binary_cross_entropy = 0.605979
Epoch 264
Loss = 5.7059e-01, PNorm = 62.1036, GNorm = 40.7020, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.616334
Epoch 265
Validation binary_cross_entropy = 0.636456
Epoch 266
Validation binary_cross_entropy = 0.652572
Epoch 267
Validation binary_cross_entropy = 0.654541
Epoch 268
Validation binary_cross_entropy = 0.640279
Epoch 269
Loss = 7.9560e-01, PNorm = 62.1086, GNorm = 52.8165, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.625065
Epoch 270
Validation binary_cross_entropy = 0.605879
Epoch 271
Validation binary_cross_entropy = 0.592079
Epoch 272
Validation binary_cross_entropy = 0.580970
Epoch 273
Validation binary_cross_entropy = 0.572136
Epoch 274
Loss = 6.6890e-01, PNorm = 62.1040, GNorm = 55.8764, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.556946
Epoch 275
Validation binary_cross_entropy = 0.544838
Epoch 276
Validation binary_cross_entropy = 0.537691
Epoch 277
Validation binary_cross_entropy = 0.525906
Epoch 278
Validation binary_cross_entropy = 0.519069
Epoch 279
Loss = 8.7187e-01, PNorm = 62.1062, GNorm = 75.2376, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.521321
Epoch 280
Validation binary_cross_entropy = 0.521669
Epoch 281
Validation binary_cross_entropy = 0.525300
Epoch 282
Validation binary_cross_entropy = 0.529008
Epoch 283
Validation binary_cross_entropy = 0.524587
Epoch 284
Loss = 1.1747e+00, PNorm = 62.1123, GNorm = 96.7075, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.529801
Epoch 285
Validation binary_cross_entropy = 0.545417
Epoch 286
Validation binary_cross_entropy = 0.562855
Epoch 287
Validation binary_cross_entropy = 0.573135
Epoch 288
Validation binary_cross_entropy = 0.587549
Epoch 289
Loss = 7.7730e-01, PNorm = 62.1183, GNorm = 49.0881, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.595031
Epoch 290
Validation binary_cross_entropy = 0.604920
Epoch 291
Validation binary_cross_entropy = 0.601180
Epoch 292
Validation binary_cross_entropy = 0.593329
Epoch 293
Validation binary_cross_entropy = 0.576802
Epoch 294
Loss = 1.9507e+00, PNorm = 62.1160, GNorm = 33.8650, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.565983
Epoch 295
Validation binary_cross_entropy = 0.563937
Epoch 296
Validation binary_cross_entropy = 0.561651
Epoch 297
Validation binary_cross_entropy = 0.561736
Epoch 298
Validation binary_cross_entropy = 0.562461
Epoch 299
Loss = 6.8002e-01, PNorm = 62.1209, GNorm = 65.6866, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.569123
Model 0 best validation binary_cross_entropy = 0.295212 on epoch 153
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.393767
Ensemble test binary_cross_entropy = 0.393767
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.910461
Epoch 1
Validation binary_cross_entropy = 89.895798
Epoch 2
Validation binary_cross_entropy = 5.195484
Epoch 3
Validation binary_cross_entropy = 6.082535
Epoch 4
Loss = 9.6816e+01, PNorm = 60.3444, GNorm = 1597.1760, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 29.033190
Epoch 5
Validation binary_cross_entropy = 27.178320
Epoch 6
Validation binary_cross_entropy = 28.962574
Epoch 7
Validation binary_cross_entropy = 7.658797
Epoch 8
Validation binary_cross_entropy = 9.215878
Epoch 9
Loss = 3.1706e+01, PNorm = 60.4776, GNorm = 887.5621, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 29.928074
Epoch 10
Validation binary_cross_entropy = 27.309036
Epoch 11
Validation binary_cross_entropy = 11.862511
Epoch 12
Validation binary_cross_entropy = 7.988871
Epoch 13
Validation binary_cross_entropy = 29.124931
Epoch 14
Loss = 3.7696e+01, PNorm = 60.6565, GNorm = 869.6079, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 33.862362
Epoch 15
Validation binary_cross_entropy = 6.519307
Epoch 16
Validation binary_cross_entropy = 6.992502
Epoch 17
Validation binary_cross_entropy = 5.212093
Epoch 18
Validation binary_cross_entropy = 35.866589
Epoch 19
Loss = 4.9264e+01, PNorm = 60.8007, GNorm = 590.9867, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 15.155524
Epoch 20
Validation binary_cross_entropy = 5.961900
Epoch 21
Validation binary_cross_entropy = 9.307093
Epoch 22
Validation binary_cross_entropy = 9.077148
Epoch 23
Validation binary_cross_entropy = 7.516445
Epoch 24
Loss = 3.8457e+01, PNorm = 60.9035, GNorm = 620.2688, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 10.079300
Epoch 25
Validation binary_cross_entropy = 26.562841
Epoch 26
Validation binary_cross_entropy = 22.638163
Epoch 27
Validation binary_cross_entropy = 6.919408
Epoch 28
Validation binary_cross_entropy = 5.696132
Epoch 29
Loss = 4.1756e+01, PNorm = 60.9775, GNorm = 767.1441, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.133799
Epoch 30
Validation binary_cross_entropy = 13.979032
Epoch 31
Validation binary_cross_entropy = 22.963793
Epoch 32
Validation binary_cross_entropy = 13.218513
Epoch 33
Validation binary_cross_entropy = 8.236827
Epoch 34
Loss = 4.4341e+01, PNorm = 61.0281, GNorm = 1101.8102, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 6.533974
Epoch 35
Validation binary_cross_entropy = 5.772882
Epoch 36
Validation binary_cross_entropy = 7.162041
Epoch 37
Validation binary_cross_entropy = 9.870926
Epoch 38
Validation binary_cross_entropy = 6.959590
Epoch 39
Loss = 2.8303e+01, PNorm = 61.0661, GNorm = 699.5509, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.424170
Epoch 40
Validation binary_cross_entropy = 4.519806
Epoch 41
Validation binary_cross_entropy = 3.963547
Epoch 42
Validation binary_cross_entropy = 3.902696
Epoch 43
Validation binary_cross_entropy = 8.158201
Epoch 44
Loss = 2.7999e+01, PNorm = 61.0989, GNorm = 601.7679, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 8.770010
Epoch 45
Validation binary_cross_entropy = 5.215508
Epoch 46
Validation binary_cross_entropy = 3.250185
Epoch 47
Validation binary_cross_entropy = 3.518401
Epoch 48
Validation binary_cross_entropy = 3.366699
Epoch 49
Loss = 1.5680e+01, PNorm = 61.1284, GNorm = 429.4712, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 2.984800
Epoch 50
Validation binary_cross_entropy = 4.599165
Epoch 51
Validation binary_cross_entropy = 11.089190
Epoch 52
Validation binary_cross_entropy = 7.437110
Epoch 53
Validation binary_cross_entropy = 3.671465
Epoch 54
Loss = 1.9829e+01, PNorm = 61.1622, GNorm = 441.6762, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 3.005742
Epoch 55
Validation binary_cross_entropy = 2.369689
Epoch 56
Validation binary_cross_entropy = 2.752715
Epoch 57
Validation binary_cross_entropy = 3.117223
Epoch 58
Validation binary_cross_entropy = 3.442181
Epoch 59
Loss = 1.8797e+01, PNorm = 61.1922, GNorm = 417.4520, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 3.374796
Epoch 60
Validation binary_cross_entropy = 2.913790
Epoch 61
Validation binary_cross_entropy = 1.934474
Epoch 62
Validation binary_cross_entropy = 1.872398
Epoch 63
Validation binary_cross_entropy = 2.256604
Epoch 64
Loss = 1.0876e+01, PNorm = 61.2331, GNorm = 587.0401, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 4.799805
Epoch 65
Validation binary_cross_entropy = 8.977406
Epoch 66
Validation binary_cross_entropy = 3.216004
Epoch 67
Validation binary_cross_entropy = 2.381397
Epoch 68
Validation binary_cross_entropy = 2.392533
Epoch 69
Loss = 1.2630e+01, PNorm = 61.2698, GNorm = 349.0837, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.192678
Epoch 70
Validation binary_cross_entropy = 4.121068
Epoch 71
Validation binary_cross_entropy = 9.052496
Epoch 72
Validation binary_cross_entropy = 10.926736
Epoch 73
Validation binary_cross_entropy = 7.288355
Epoch 74
Loss = 8.9508e+00, PNorm = 61.3097, GNorm = 322.4958, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 6.937002
Epoch 75
Validation binary_cross_entropy = 3.172722
Epoch 76
Validation binary_cross_entropy = 1.886137
Epoch 77
Validation binary_cross_entropy = 1.984681
Epoch 78
Validation binary_cross_entropy = 1.404915
Epoch 79
Loss = 8.3684e+00, PNorm = 61.3546, GNorm = 267.7836, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.101393
Epoch 80
Validation binary_cross_entropy = 2.049267
Epoch 81
Validation binary_cross_entropy = 8.039883
Epoch 82
Validation binary_cross_entropy = 9.596270
Epoch 83
Validation binary_cross_entropy = 3.400951
Epoch 84
Loss = 1.2036e+01, PNorm = 61.4010, GNorm = 398.4106, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.978577
Epoch 85
Validation binary_cross_entropy = 1.518086
Epoch 86
Validation binary_cross_entropy = 1.513995
Epoch 87
Validation binary_cross_entropy = 1.315973
Epoch 88
Validation binary_cross_entropy = 1.223417
Epoch 89
Loss = 3.8623e+00, PNorm = 61.4270, GNorm = 214.1844, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.948835
Epoch 90
Validation binary_cross_entropy = 1.924256
Epoch 91
Validation binary_cross_entropy = 1.335160
Epoch 92
Validation binary_cross_entropy = 1.843495
Epoch 93
Validation binary_cross_entropy = 1.103132
Epoch 94
Loss = 6.4188e+00, PNorm = 61.4525, GNorm = 261.2389, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.937169
Epoch 95
Validation binary_cross_entropy = 0.820985
Epoch 96
Validation binary_cross_entropy = 0.821227
Epoch 97
Validation binary_cross_entropy = 0.917409
Epoch 98
Validation binary_cross_entropy = 0.949348
Epoch 99
Loss = 5.6851e+00, PNorm = 61.4809, GNorm = 355.5816, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.871593
Epoch 100
Validation binary_cross_entropy = 0.756652
Epoch 101
Validation binary_cross_entropy = 0.904203
Epoch 102
Validation binary_cross_entropy = 0.883512
Epoch 103
Validation binary_cross_entropy = 0.719043
Epoch 104
Loss = 6.9516e+00, PNorm = 61.4911, GNorm = 199.0882, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.731580
Epoch 105
Validation binary_cross_entropy = 0.703594
Epoch 106
Validation binary_cross_entropy = 0.613143
Epoch 107
Validation binary_cross_entropy = 0.962323
Epoch 108
Validation binary_cross_entropy = 3.680774
Epoch 109
Loss = 4.1430e+00, PNorm = 61.4989, GNorm = 174.0843, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 6.669466
Epoch 110
Validation binary_cross_entropy = 6.002373
Epoch 111
Validation binary_cross_entropy = 3.753310
Epoch 112
Validation binary_cross_entropy = 1.231919
Epoch 113
Validation binary_cross_entropy = 0.749468
Epoch 114
Loss = 6.2780e+00, PNorm = 61.5155, GNorm = 149.4437, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.537313
Epoch 115
Validation binary_cross_entropy = 0.537784
Epoch 116
Validation binary_cross_entropy = 0.627478
Epoch 117
Validation binary_cross_entropy = 1.264386
Epoch 118
Validation binary_cross_entropy = 3.028074
Epoch 119
Loss = 5.1292e+00, PNorm = 61.5328, GNorm = 180.4421, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 5.172270
Epoch 120
Validation binary_cross_entropy = 5.218568
Epoch 121
Validation binary_cross_entropy = 3.060813
Epoch 122
Validation binary_cross_entropy = 1.173849
Epoch 123
Validation binary_cross_entropy = 0.639115
Epoch 124
Loss = 5.0331e+00, PNorm = 61.5548, GNorm = 270.7294, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.496845
Epoch 125
Validation binary_cross_entropy = 0.470046
Epoch 126
Validation binary_cross_entropy = 0.529653
Epoch 127
Validation binary_cross_entropy = 0.595452
Epoch 128
Validation binary_cross_entropy = 0.602511
Epoch 129
Loss = 5.6818e+00, PNorm = 61.5709, GNorm = 173.5619, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.590942
Epoch 130
Validation binary_cross_entropy = 0.631148
Epoch 131
Validation binary_cross_entropy = 0.625970
Epoch 132
Validation binary_cross_entropy = 0.662170
Epoch 133
Validation binary_cross_entropy = 0.773829
Epoch 134
Loss = 4.7699e+00, PNorm = 61.5840, GNorm = 150.1023, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.972394
Epoch 135
Validation binary_cross_entropy = 0.817437
Epoch 136
Validation binary_cross_entropy = 0.583004
Epoch 137
Validation binary_cross_entropy = 0.487031
Epoch 138
Validation binary_cross_entropy = 0.516716
Epoch 139
Loss = 6.7343e+00, PNorm = 61.6286, GNorm = 251.1501, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.559104
Epoch 140
Validation binary_cross_entropy = 0.617887
Epoch 141
Validation binary_cross_entropy = 0.632447
Epoch 142
Validation binary_cross_entropy = 0.682885
Epoch 143
Validation binary_cross_entropy = 0.834471
Epoch 144
Loss = 3.1590e+00, PNorm = 61.6560, GNorm = 120.2630, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.016838
Epoch 145
Validation binary_cross_entropy = 0.939171
Epoch 146
Validation binary_cross_entropy = 0.659006
Epoch 147
Validation binary_cross_entropy = 0.443369
Epoch 148
Validation binary_cross_entropy = 0.360122
Epoch 149
Loss = 3.4174e+00, PNorm = 61.6771, GNorm = 113.6468, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.358830
Epoch 150
Validation binary_cross_entropy = 0.363193
Epoch 151
Validation binary_cross_entropy = 0.405193
Epoch 152
Validation binary_cross_entropy = 0.486046
Epoch 153
Validation binary_cross_entropy = 0.692800
Epoch 154
Loss = 4.6378e+00, PNorm = 61.6795, GNorm = 319.5310, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.739407
Epoch 155
Validation binary_cross_entropy = 0.671535
Epoch 156
Validation binary_cross_entropy = 0.598235
Epoch 157
Validation binary_cross_entropy = 0.567952
Epoch 158
Validation binary_cross_entropy = 0.452055
Epoch 159
Loss = 4.6007e+00, PNorm = 61.6887, GNorm = 260.7380, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.340137
Epoch 160
Validation binary_cross_entropy = 0.307466
Epoch 161
Validation binary_cross_entropy = 0.308721
Epoch 162
Validation binary_cross_entropy = 0.329268
Epoch 163
Validation binary_cross_entropy = 0.461449
Epoch 164
Loss = 2.0298e+00, PNorm = 61.6968, GNorm = 108.0389, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.599268
Epoch 165
Validation binary_cross_entropy = 0.678533
Epoch 166
Validation binary_cross_entropy = 0.769666
Epoch 167
Validation binary_cross_entropy = 0.777517
Epoch 168
Validation binary_cross_entropy = 0.599744
Epoch 169
Loss = 2.2932e+00, PNorm = 61.7091, GNorm = 182.1509, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.526754
Epoch 170
Validation binary_cross_entropy = 0.478539
Epoch 171
Validation binary_cross_entropy = 0.433830
Epoch 172
Validation binary_cross_entropy = 0.453329
Epoch 173
Validation binary_cross_entropy = 0.470808
Epoch 174
Loss = 2.1531e+00, PNorm = 61.7269, GNorm = 128.3040, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.554518
Epoch 175
Validation binary_cross_entropy = 0.697106
Epoch 176
Validation binary_cross_entropy = 0.653479
Epoch 177
Validation binary_cross_entropy = 0.576324
Epoch 178
Validation binary_cross_entropy = 0.541537
Epoch 179
Loss = 1.8344e+00, PNorm = 61.7448, GNorm = 130.2971, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.546452
Epoch 180
Validation binary_cross_entropy = 0.560252
Epoch 181
Validation binary_cross_entropy = 0.602156
Epoch 182
Validation binary_cross_entropy = 0.632660
Epoch 183
Validation binary_cross_entropy = 0.612001
Epoch 184
Loss = 5.8417e+00, PNorm = 61.7565, GNorm = 277.9989, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.512976
Epoch 185
Validation binary_cross_entropy = 0.423827
Epoch 186
Validation binary_cross_entropy = 0.370928
Epoch 187
Validation binary_cross_entropy = 0.360781
Epoch 188
Validation binary_cross_entropy = 0.387136
Epoch 189
Loss = 1.9225e+00, PNorm = 61.7721, GNorm = 79.9861, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.449218
Epoch 190
Validation binary_cross_entropy = 0.518613
Epoch 191
Validation binary_cross_entropy = 0.582577
Epoch 192
Validation binary_cross_entropy = 0.552365
Epoch 193
Validation binary_cross_entropy = 0.531794
Epoch 194
Loss = 1.5482e+00, PNorm = 61.7874, GNorm = 92.9556, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.498136
Epoch 195
Validation binary_cross_entropy = 0.508198
Epoch 196
Validation binary_cross_entropy = 0.524630
Epoch 197
Validation binary_cross_entropy = 0.543752
Epoch 198
Validation binary_cross_entropy = 0.559716
Epoch 199
Loss = 2.4036e+00, PNorm = 61.7906, GNorm = 185.9738, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.548222
Epoch 200
Validation binary_cross_entropy = 0.502352
Epoch 201
Validation binary_cross_entropy = 0.434038
Epoch 202
Validation binary_cross_entropy = 0.405210
Epoch 203
Validation binary_cross_entropy = 0.420369
Epoch 204
Loss = 1.5336e+00, PNorm = 61.8012, GNorm = 110.2360, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.488383
Epoch 205
Validation binary_cross_entropy = 0.619255
Epoch 206
Validation binary_cross_entropy = 0.749956
Epoch 207
Validation binary_cross_entropy = 0.865494
Epoch 208
Validation binary_cross_entropy = 0.847619
Epoch 209
Loss = 1.5519e+00, PNorm = 61.8345, GNorm = 112.4017, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.651841
Epoch 210
Validation binary_cross_entropy = 0.459209
Epoch 211
Validation binary_cross_entropy = 0.346005
Epoch 212
Validation binary_cross_entropy = 0.322106
Epoch 213
Validation binary_cross_entropy = 0.340624
Epoch 214
Loss = 1.3425e+00, PNorm = 61.8476, GNorm = 117.4915, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.403091
Epoch 215
Validation binary_cross_entropy = 0.489047
Epoch 216
Validation binary_cross_entropy = 0.583440
Epoch 217
Validation binary_cross_entropy = 0.604085
Epoch 218
Validation binary_cross_entropy = 0.606726
Epoch 219
Loss = 1.1648e+00, PNorm = 61.8419, GNorm = 58.7436, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.592357
Epoch 220
Validation binary_cross_entropy = 0.571239
Epoch 221
Validation binary_cross_entropy = 0.546943
Epoch 222
Validation binary_cross_entropy = 0.527960
Epoch 223
Validation binary_cross_entropy = 0.517103
Epoch 224
Loss = 1.2238e+00, PNorm = 61.8455, GNorm = 114.5323, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.511455
Epoch 225
Validation binary_cross_entropy = 0.523672
Epoch 226
Validation binary_cross_entropy = 0.538946
Epoch 227
Validation binary_cross_entropy = 0.574019
Epoch 228
Validation binary_cross_entropy = 0.610239
Epoch 229
Loss = 1.1254e+00, PNorm = 61.8607, GNorm = 67.9549, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.635667
Epoch 230
Validation binary_cross_entropy = 0.635222
Epoch 231
Validation binary_cross_entropy = 0.635043
Epoch 232
Validation binary_cross_entropy = 0.636213
Epoch 233
Validation binary_cross_entropy = 0.607399
Epoch 234
Loss = 1.5072e+00, PNorm = 61.8771, GNorm = 103.7428, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.581478
Epoch 235
Validation binary_cross_entropy = 0.553008
Epoch 236
Validation binary_cross_entropy = 0.526375
Epoch 237
Validation binary_cross_entropy = 0.498929
Epoch 238
Validation binary_cross_entropy = 0.491261
Epoch 239
Loss = 1.1577e+00, PNorm = 61.8904, GNorm = 73.0293, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.512077
Epoch 240
Validation binary_cross_entropy = 0.587390
Epoch 241
Validation binary_cross_entropy = 0.630777
Epoch 242
Validation binary_cross_entropy = 0.594031
Epoch 243
Validation binary_cross_entropy = 0.555170
Epoch 244
Loss = 9.9526e-01, PNorm = 61.8992, GNorm = 44.0684, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.536289
Epoch 245
Validation binary_cross_entropy = 0.516475
Epoch 246
Validation binary_cross_entropy = 0.519006
Epoch 247
Validation binary_cross_entropy = 0.536742
Epoch 248
Validation binary_cross_entropy = 0.567295
Epoch 249
Loss = 8.8133e-01, PNorm = 61.9265, GNorm = 56.1373, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.603182
Epoch 250
Validation binary_cross_entropy = 0.638406
Epoch 251
Validation binary_cross_entropy = 0.642407
Epoch 252
Validation binary_cross_entropy = 0.624930
Epoch 253
Validation binary_cross_entropy = 0.626211
Epoch 254
Loss = 9.3253e-01, PNorm = 61.9560, GNorm = 62.5283, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.611102
Epoch 255
Validation binary_cross_entropy = 0.598403
Epoch 256
Validation binary_cross_entropy = 0.576011
Epoch 257
Validation binary_cross_entropy = 0.556731
Epoch 258
Validation binary_cross_entropy = 0.577431
Epoch 259
Loss = 1.1978e+00, PNorm = 61.9691, GNorm = 67.4652, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.591887
Epoch 260
Validation binary_cross_entropy = 0.606853
Epoch 261
Validation binary_cross_entropy = 0.570969
Epoch 262
Validation binary_cross_entropy = 0.543172
Epoch 263
Validation binary_cross_entropy = 0.518456
Epoch 264
Loss = 8.5203e-01, PNorm = 61.9853, GNorm = 79.4775, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.604704
Epoch 265
Validation binary_cross_entropy = 0.643540
Epoch 266
Validation binary_cross_entropy = 0.643407
Epoch 267
Validation binary_cross_entropy = 0.594333
Epoch 268
Validation binary_cross_entropy = 0.536369
Epoch 269
Loss = 8.0342e-01, PNorm = 61.9973, GNorm = 61.6735, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.501290
Epoch 270
Validation binary_cross_entropy = 0.494550
Epoch 271
Validation binary_cross_entropy = 0.513351
Epoch 272
Validation binary_cross_entropy = 0.524016
Epoch 273
Validation binary_cross_entropy = 0.539912
Epoch 274
Loss = 9.2935e-01, PNorm = 61.9892, GNorm = 99.8778, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.546260
Epoch 275
Validation binary_cross_entropy = 0.542967
Epoch 276
Validation binary_cross_entropy = 0.561106
Epoch 277
Validation binary_cross_entropy = 0.594301
Epoch 278
Validation binary_cross_entropy = 0.615480
Epoch 279
Loss = 8.6732e-01, PNorm = 61.9979, GNorm = 45.9861, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.610864
Epoch 280
Validation binary_cross_entropy = 0.600505
Epoch 281
Validation binary_cross_entropy = 0.596055
Epoch 282
Validation binary_cross_entropy = 0.599419
Epoch 283
Validation binary_cross_entropy = 0.617144
Epoch 284
Loss = 7.9822e-01, PNorm = 62.0038, GNorm = 92.8545, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.621372
Epoch 285
Validation binary_cross_entropy = 0.598851
Epoch 286
Validation binary_cross_entropy = 0.559997
Epoch 287
Validation binary_cross_entropy = 0.527504
Epoch 288
Validation binary_cross_entropy = 0.514206
Epoch 289
Loss = 8.6551e-01, PNorm = 62.0083, GNorm = 68.2429, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.490210
Epoch 290
Validation binary_cross_entropy = 0.472857
Epoch 291
Validation binary_cross_entropy = 0.464549
Epoch 292
Validation binary_cross_entropy = 0.469926
Epoch 293
Validation binary_cross_entropy = 0.485422
Epoch 294
Loss = 1.0730e+00, PNorm = 62.0141, GNorm = 65.4264, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.502862
Epoch 295
Validation binary_cross_entropy = 0.520610
Epoch 296
Validation binary_cross_entropy = 0.543909
Epoch 297
Validation binary_cross_entropy = 0.550287
Epoch 298
Validation binary_cross_entropy = 0.562548
Epoch 299
Loss = 8.4951e-01, PNorm = 62.0290, GNorm = 96.4152, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.565531
Model 0 best validation binary_cross_entropy = 0.307466 on epoch 160
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.369676
Ensemble test binary_cross_entropy = 0.369676
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.906044
Epoch 1
Validation binary_cross_entropy = 87.692741
Epoch 2
Validation binary_cross_entropy = 4.547402
Epoch 3
Validation binary_cross_entropy = 6.287181
Epoch 4
Loss = 7.5537e+01, PNorm = 60.3440, GNorm = 1436.8173, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 12.300276
Epoch 5
Validation binary_cross_entropy = 9.066806
Epoch 6
Validation binary_cross_entropy = 12.735362
Epoch 7
Validation binary_cross_entropy = 20.403761
Epoch 8
Validation binary_cross_entropy = 12.777194
Epoch 9
Loss = 6.2959e+01, PNorm = 60.4708, GNorm = 1289.1667, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 6.680998
Epoch 10
Validation binary_cross_entropy = 5.487396
Epoch 11
Validation binary_cross_entropy = 9.125198
Epoch 12
Validation binary_cross_entropy = 10.742023
Epoch 13
Validation binary_cross_entropy = 7.992140
Epoch 14
Loss = 3.6511e+01, PNorm = 60.6436, GNorm = 878.5912, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 5.347943
Epoch 15
Validation binary_cross_entropy = 6.969142
Epoch 16
Validation binary_cross_entropy = 12.798553
Epoch 17
Validation binary_cross_entropy = 9.904370
Epoch 18
Validation binary_cross_entropy = 6.169961
Epoch 19
Loss = 7.5687e+01, PNorm = 60.8298, GNorm = 1514.8063, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 8.257014
Epoch 20
Validation binary_cross_entropy = 7.245559
Epoch 21
Validation binary_cross_entropy = 7.416697
Epoch 22
Validation binary_cross_entropy = 10.328136
Epoch 23
Validation binary_cross_entropy = 11.786729
Epoch 24
Loss = 4.2176e+01, PNorm = 60.9826, GNorm = 578.5338, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 7.058319
Epoch 25
Validation binary_cross_entropy = 10.023672
Epoch 26
Validation binary_cross_entropy = 23.708189
Epoch 27
Validation binary_cross_entropy = 7.193981
Epoch 28
Validation binary_cross_entropy = 6.190037
Epoch 29
Loss = 2.9986e+01, PNorm = 61.1191, GNorm = 581.0945, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 4.525874
Epoch 30
Validation binary_cross_entropy = 10.090301
Epoch 31
Validation binary_cross_entropy = 9.639002
Epoch 32
Validation binary_cross_entropy = 4.194580
Epoch 33
Validation binary_cross_entropy = 3.401764
Epoch 34
Loss = 2.7152e+01, PNorm = 61.1834, GNorm = 642.1315, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 3.274876
Epoch 35
Validation binary_cross_entropy = 3.578022
Epoch 36
Validation binary_cross_entropy = 2.936430
Epoch 37
Validation binary_cross_entropy = 5.928451
Epoch 38
Validation binary_cross_entropy = 7.847784
Epoch 39
Loss = 2.1287e+01, PNorm = 61.2564, GNorm = 569.1603, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 6.918738
Epoch 40
Validation binary_cross_entropy = 18.477325
Epoch 41
Validation binary_cross_entropy = 21.856552
Epoch 42
Validation binary_cross_entropy = 12.376894
Epoch 43
Validation binary_cross_entropy = 3.724127
Epoch 44
Loss = 1.4497e+01, PNorm = 61.3149, GNorm = 491.9507, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 2.884974
Epoch 45
Validation binary_cross_entropy = 4.428637
Epoch 46
Validation binary_cross_entropy = 9.955294
Epoch 47
Validation binary_cross_entropy = 6.671346
Epoch 48
Validation binary_cross_entropy = 2.824865
Epoch 49
Loss = 2.0733e+01, PNorm = 61.3806, GNorm = 466.5818, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.005259
Epoch 50
Validation binary_cross_entropy = 4.052434
Epoch 51
Validation binary_cross_entropy = 21.969814
Epoch 52
Validation binary_cross_entropy = 14.419149
Epoch 53
Validation binary_cross_entropy = 2.137799
Epoch 54
Loss = 8.4404e+00, PNorm = 61.3995, GNorm = 341.9456, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.878108
Epoch 55
Validation binary_cross_entropy = 2.929149
Epoch 56
Validation binary_cross_entropy = 3.289028
Epoch 57
Validation binary_cross_entropy = 6.967175
Epoch 58
Validation binary_cross_entropy = 12.799664
Epoch 59
Loss = 1.2802e+01, PNorm = 61.4251, GNorm = 277.6077, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 16.341936
Epoch 60
Validation binary_cross_entropy = 9.104646
Epoch 61
Validation binary_cross_entropy = 2.933504
Epoch 62
Validation binary_cross_entropy = 2.789342
Epoch 63
Validation binary_cross_entropy = 2.620061
Epoch 64
Loss = 9.6290e+00, PNorm = 61.4535, GNorm = 275.4234, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 12.179354
Epoch 65
Validation binary_cross_entropy = 18.269510
Epoch 66
Validation binary_cross_entropy = 8.750057
Epoch 67
Validation binary_cross_entropy = 2.931735
Epoch 68
Validation binary_cross_entropy = 2.128896
Epoch 69
Loss = 6.2599e+00, PNorm = 61.4806, GNorm = 360.7792, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.522806
Epoch 70
Validation binary_cross_entropy = 1.658501
Epoch 71
Validation binary_cross_entropy = 3.063609
Epoch 72
Validation binary_cross_entropy = 6.022708
Epoch 73
Validation binary_cross_entropy = 5.868086
Epoch 74
Loss = 1.1496e+01, PNorm = 61.5012, GNorm = 392.4557, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.250346
Epoch 75
Validation binary_cross_entropy = 1.834269
Epoch 76
Validation binary_cross_entropy = 1.966408
Epoch 77
Validation binary_cross_entropy = 2.197073
Epoch 78
Validation binary_cross_entropy = 1.617246
Epoch 79
Loss = 7.7986e+00, PNorm = 61.5323, GNorm = 244.9639, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.604468
Epoch 80
Validation binary_cross_entropy = 2.792469
Epoch 81
Validation binary_cross_entropy = 8.702483
Epoch 82
Validation binary_cross_entropy = 8.995066
Epoch 83
Validation binary_cross_entropy = 3.979714
Epoch 84
Loss = 1.0311e+01, PNorm = 61.5643, GNorm = 361.4752, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.611207
Epoch 85
Validation binary_cross_entropy = 1.587319
Epoch 86
Validation binary_cross_entropy = 1.209150
Epoch 87
Validation binary_cross_entropy = 1.412025
Epoch 88
Validation binary_cross_entropy = 1.496488
Epoch 89
Loss = 1.0029e+01, PNorm = 61.5912, GNorm = 289.1618, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.908692
Epoch 90
Validation binary_cross_entropy = 1.640754
Epoch 91
Validation binary_cross_entropy = 2.976327
Epoch 92
Validation binary_cross_entropy = 3.573407
Epoch 93
Validation binary_cross_entropy = 3.676384
Epoch 94
Loss = 7.1109e+00, PNorm = 61.6191, GNorm = 282.4290, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 4.112824
Epoch 95
Validation binary_cross_entropy = 3.492044
Epoch 96
Validation binary_cross_entropy = 2.014070
Epoch 97
Validation binary_cross_entropy = 1.311312
Epoch 98
Validation binary_cross_entropy = 1.451512
Epoch 99
Loss = 8.2740e+00, PNorm = 61.6573, GNorm = 276.1824, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.258990
Epoch 100
Validation binary_cross_entropy = 1.046071
Epoch 101
Validation binary_cross_entropy = 0.849231
Epoch 102
Validation binary_cross_entropy = 0.739849
Epoch 103
Validation binary_cross_entropy = 0.813720
Epoch 104
Loss = 5.6569e+00, PNorm = 61.6954, GNorm = 253.3433, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.196070
Epoch 105
Validation binary_cross_entropy = 2.275146
Epoch 106
Validation binary_cross_entropy = 1.023882
Epoch 107
Validation binary_cross_entropy = 0.711374
Epoch 108
Validation binary_cross_entropy = 0.711299
Epoch 109
Loss = 6.9162e+00, PNorm = 61.7492, GNorm = 235.0547, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.800703
Epoch 110
Validation binary_cross_entropy = 0.842086
Epoch 111
Validation binary_cross_entropy = 0.651403
Epoch 112
Validation binary_cross_entropy = 0.750548
Epoch 113
Validation binary_cross_entropy = 0.975028
Epoch 114
Loss = 1.1144e+01, PNorm = 61.7496, GNorm = 189.8633, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.725708
Epoch 115
Validation binary_cross_entropy = 0.617379
Epoch 116
Validation binary_cross_entropy = 0.586202
Epoch 117
Validation binary_cross_entropy = 0.602568
Epoch 118
Validation binary_cross_entropy = 0.629410
Epoch 119
Loss = 6.0694e+00, PNorm = 61.7635, GNorm = 288.7894, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.698413
Epoch 120
Validation binary_cross_entropy = 0.741859
Epoch 121
Validation binary_cross_entropy = 0.611441
Epoch 122
Validation binary_cross_entropy = 0.553242
Epoch 123
Validation binary_cross_entropy = 0.517181
Epoch 124
Loss = 7.2265e+00, PNorm = 61.7851, GNorm = 179.5456, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.509767
Epoch 125
Validation binary_cross_entropy = 0.516695
Epoch 126
Validation binary_cross_entropy = 0.531499
Epoch 127
Validation binary_cross_entropy = 0.631653
Epoch 128
Validation binary_cross_entropy = 0.825843
Epoch 129
Loss = 3.9059e+00, PNorm = 61.7908, GNorm = 154.5555, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.152923
Epoch 130
Validation binary_cross_entropy = 1.084811
Epoch 131
Validation binary_cross_entropy = 0.764707
Epoch 132
Validation binary_cross_entropy = 0.544463
Epoch 133
Validation binary_cross_entropy = 0.494784
Epoch 134
Loss = 3.0217e+00, PNorm = 61.7990, GNorm = 156.9739, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.486668
Epoch 135
Validation binary_cross_entropy = 0.534469
Epoch 136
Validation binary_cross_entropy = 0.735370
Epoch 137
Validation binary_cross_entropy = 1.186755
Epoch 138
Validation binary_cross_entropy = 1.387006
Epoch 139
Loss = 3.9526e+00, PNorm = 61.8190, GNorm = 248.7921, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.915915
Epoch 140
Validation binary_cross_entropy = 0.663772
Epoch 141
Validation binary_cross_entropy = 0.549689
Epoch 142
Validation binary_cross_entropy = 0.487319
Epoch 143
Validation binary_cross_entropy = 0.450928
Epoch 144
Loss = 3.5613e+00, PNorm = 61.8493, GNorm = 278.5678, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.416807
Epoch 145
Validation binary_cross_entropy = 0.415940
Epoch 146
Validation binary_cross_entropy = 0.420453
Epoch 147
Validation binary_cross_entropy = 0.444790
Epoch 148
Validation binary_cross_entropy = 0.483653
Epoch 149
Loss = 4.1710e+00, PNorm = 61.8619, GNorm = 332.6906, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.509513
Epoch 150
Validation binary_cross_entropy = 0.469092
Epoch 151
Validation binary_cross_entropy = 0.455191
Epoch 152
Validation binary_cross_entropy = 0.457774
Epoch 153
Validation binary_cross_entropy = 0.453792
Epoch 154
Loss = 4.2067e+00, PNorm = 61.8651, GNorm = 171.3645, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.451183
Epoch 155
Validation binary_cross_entropy = 0.462983
Epoch 156
Validation binary_cross_entropy = 0.471533
Epoch 157
Validation binary_cross_entropy = 0.469649
Epoch 158
Validation binary_cross_entropy = 0.450585
Epoch 159
Loss = 2.7960e+00, PNorm = 61.8798, GNorm = 108.7555, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.432413
Epoch 160
Validation binary_cross_entropy = 0.440469
Epoch 161
Validation binary_cross_entropy = 0.439416
Epoch 162
Validation binary_cross_entropy = 0.446648
Epoch 163
Validation binary_cross_entropy = 0.489683
Epoch 164
Loss = 2.6090e+00, PNorm = 61.9014, GNorm = 123.1936, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.545689
Epoch 165
Validation binary_cross_entropy = 0.557507
Epoch 166
Validation binary_cross_entropy = 0.554545
Epoch 167
Validation binary_cross_entropy = 0.547924
Epoch 168
Validation binary_cross_entropy = 0.558128
Epoch 169
Loss = 3.3229e+00, PNorm = 61.9240, GNorm = 163.1934, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.567993
Epoch 170
Validation binary_cross_entropy = 0.575406
Epoch 171
Validation binary_cross_entropy = 0.578802
Epoch 172
Validation binary_cross_entropy = 0.575703
Epoch 173
Validation binary_cross_entropy = 0.603370
Epoch 174
Loss = 2.0363e+00, PNorm = 61.9359, GNorm = 58.8906, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.621796
Epoch 175
Validation binary_cross_entropy = 0.644226
Epoch 176
Validation binary_cross_entropy = 0.631885
Epoch 177
Validation binary_cross_entropy = 0.598465
Epoch 178
Validation binary_cross_entropy = 0.562260
Epoch 179
Loss = 3.1848e+00, PNorm = 61.9420, GNorm = 158.0239, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.517019
Epoch 180
Validation binary_cross_entropy = 0.469571
Epoch 181
Validation binary_cross_entropy = 0.434967
Epoch 182
Validation binary_cross_entropy = 0.392308
Epoch 183
Validation binary_cross_entropy = 0.410127
Epoch 184
Loss = 2.1279e+00, PNorm = 61.9642, GNorm = 138.4376, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.469230
Epoch 185
Validation binary_cross_entropy = 0.531738
Epoch 186
Validation binary_cross_entropy = 0.535990
Epoch 187
Validation binary_cross_entropy = 0.542495
Epoch 188
Validation binary_cross_entropy = 0.557394
Epoch 189
Loss = 1.4987e+00, PNorm = 61.9692, GNorm = 95.8655, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.497460
Epoch 190
Validation binary_cross_entropy = 0.436097
Epoch 191
Validation binary_cross_entropy = 0.393188
Epoch 192
Validation binary_cross_entropy = 0.378170
Epoch 193
Validation binary_cross_entropy = 0.381544
Epoch 194
Loss = 1.7940e+00, PNorm = 61.9711, GNorm = 133.1448, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.385718
Epoch 195
Validation binary_cross_entropy = 0.385816
Epoch 196
Validation binary_cross_entropy = 0.384409
Epoch 197
Validation binary_cross_entropy = 0.384416
Epoch 198
Validation binary_cross_entropy = 0.390032
Epoch 199
Loss = 1.6572e+00, PNorm = 61.9797, GNorm = 80.9905, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.397227
Epoch 200
Validation binary_cross_entropy = 0.415132
Epoch 201
Validation binary_cross_entropy = 0.423843
Epoch 202
Validation binary_cross_entropy = 0.420096
Epoch 203
Validation binary_cross_entropy = 0.419246
Epoch 204
Loss = 2.3791e+00, PNorm = 62.0019, GNorm = 73.0654, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.400218
Epoch 205
Validation binary_cross_entropy = 0.373600
Epoch 206
Validation binary_cross_entropy = 0.341455
Epoch 207
Validation binary_cross_entropy = 0.326008
Epoch 208
Validation binary_cross_entropy = 0.324866
Epoch 209
Loss = 1.1058e+00, PNorm = 62.0244, GNorm = 81.9564, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.344924
Epoch 210
Validation binary_cross_entropy = 0.367586
Epoch 211
Validation binary_cross_entropy = 0.367984
Epoch 212
Validation binary_cross_entropy = 0.363552
Epoch 213
Validation binary_cross_entropy = 0.357296
Epoch 214
Loss = 2.3627e+00, PNorm = 62.0306, GNorm = 69.1026, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.353485
Epoch 215
Validation binary_cross_entropy = 0.362268
Epoch 216
Validation binary_cross_entropy = 0.393892
Epoch 217
Validation binary_cross_entropy = 0.440241
Epoch 218
Validation binary_cross_entropy = 0.491624
Epoch 219
Loss = 1.3007e+00, PNorm = 62.0363, GNorm = 56.1737, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.538218
Epoch 220
Validation binary_cross_entropy = 0.561835
Epoch 221
Validation binary_cross_entropy = 0.570076
Epoch 222
Validation binary_cross_entropy = 0.561082
Epoch 223
Validation binary_cross_entropy = 0.558913
Epoch 224
Loss = 1.5905e+00, PNorm = 62.0394, GNorm = 97.2518, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.549047
Epoch 225
Validation binary_cross_entropy = 0.533984
Epoch 226
Validation binary_cross_entropy = 0.525179
Epoch 227
Validation binary_cross_entropy = 0.520126
Epoch 228
Validation binary_cross_entropy = 0.505682
Epoch 229
Loss = 1.0031e+00, PNorm = 62.0480, GNorm = 71.1203, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.507652
Epoch 230
Validation binary_cross_entropy = 0.518479
Epoch 231
Validation binary_cross_entropy = 0.540353
Epoch 232
Validation binary_cross_entropy = 0.542568
Epoch 233
Validation binary_cross_entropy = 0.532734
Epoch 234
Loss = 1.1383e+00, PNorm = 62.0565, GNorm = 95.5508, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.502418
Epoch 235
Validation binary_cross_entropy = 0.484606
Epoch 236
Validation binary_cross_entropy = 0.485800
Epoch 237
Validation binary_cross_entropy = 0.488597
Epoch 238
Validation binary_cross_entropy = 0.502558
Epoch 239
Loss = 1.1315e+00, PNorm = 62.0745, GNorm = 112.3681, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.519782
Epoch 240
Validation binary_cross_entropy = 0.511322
Epoch 241
Validation binary_cross_entropy = 0.509020
Epoch 242
Validation binary_cross_entropy = 0.507718
Epoch 243
Validation binary_cross_entropy = 0.506224
Epoch 244
Loss = 7.9884e-01, PNorm = 62.0868, GNorm = 37.4589, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.495690
Epoch 245
Validation binary_cross_entropy = 0.475029
Epoch 246
Validation binary_cross_entropy = 0.446172
Epoch 247
Validation binary_cross_entropy = 0.439369
Epoch 248
Validation binary_cross_entropy = 0.448182
Epoch 249
Loss = 1.1031e+00, PNorm = 62.1105, GNorm = 77.0799, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.453266
Epoch 250
Validation binary_cross_entropy = 0.477547
Epoch 251
Validation binary_cross_entropy = 0.478762
Epoch 252
Validation binary_cross_entropy = 0.482899
Epoch 253
Validation binary_cross_entropy = 0.483439
Epoch 254
Loss = 1.0990e+00, PNorm = 62.1119, GNorm = 63.7216, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.489328
Epoch 255
Validation binary_cross_entropy = 0.503460
Epoch 256
Validation binary_cross_entropy = 0.513998
Epoch 257
Validation binary_cross_entropy = 0.506647
Epoch 258
Validation binary_cross_entropy = 0.495398
Epoch 259
Loss = 8.9322e-01, PNorm = 62.1045, GNorm = 92.9540, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.481019
Epoch 260
Validation binary_cross_entropy = 0.453210
Epoch 261
Validation binary_cross_entropy = 0.450978
Epoch 262
Validation binary_cross_entropy = 0.447584
Epoch 263
Validation binary_cross_entropy = 0.466225
Epoch 264
Loss = 1.0749e+00, PNorm = 62.1331, GNorm = 57.5408, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.509565
Epoch 265
Validation binary_cross_entropy = 0.553035
Epoch 266
Validation binary_cross_entropy = 0.589775
Epoch 267
Validation binary_cross_entropy = 0.577074
Epoch 268
Validation binary_cross_entropy = 0.550359
Epoch 269
Loss = 1.0137e+00, PNorm = 62.1518, GNorm = 57.7466, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.533808
Epoch 270
Validation binary_cross_entropy = 0.547024
Epoch 271
Validation binary_cross_entropy = 0.536447
Epoch 272
Validation binary_cross_entropy = 0.530730
Epoch 273
Validation binary_cross_entropy = 0.531940
Epoch 274
Loss = 8.4903e-01, PNorm = 62.1539, GNorm = 73.8671, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.521857
Epoch 275
Validation binary_cross_entropy = 0.514691
Epoch 276
Validation binary_cross_entropy = 0.517876
Epoch 277
Validation binary_cross_entropy = 0.530852
Epoch 278
Validation binary_cross_entropy = 0.554441
Epoch 279
Loss = 6.5661e-01, PNorm = 62.1472, GNorm = 49.8531, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.569079
Epoch 280
Validation binary_cross_entropy = 0.583314
Epoch 281
Validation binary_cross_entropy = 0.579984
Epoch 282
Validation binary_cross_entropy = 0.568468
Epoch 283
Validation binary_cross_entropy = 0.552308
Epoch 284
Loss = 1.1114e+00, PNorm = 62.1517, GNorm = 76.8745, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.538208
Epoch 285
Validation binary_cross_entropy = 0.528606
Epoch 286
Validation binary_cross_entropy = 0.513059
Epoch 287
Validation binary_cross_entropy = 0.509099
Epoch 288
Validation binary_cross_entropy = 0.518984
Epoch 289
Loss = 8.7266e-01, PNorm = 62.1589, GNorm = 38.0385, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.528026
Epoch 290
Validation binary_cross_entropy = 0.524222
Epoch 291
Validation binary_cross_entropy = 0.528093
Epoch 292
Validation binary_cross_entropy = 0.498491
Epoch 293
Validation binary_cross_entropy = 0.466878
Epoch 294
Loss = 1.0744e+00, PNorm = 62.1558, GNorm = 84.2980, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.448195
Epoch 295
Validation binary_cross_entropy = 0.449173
Epoch 296
Validation binary_cross_entropy = 0.440260
Epoch 297
Validation binary_cross_entropy = 0.429498
Epoch 298
Validation binary_cross_entropy = 0.428815
Epoch 299
Loss = 6.4874e-01, PNorm = 62.1600, GNorm = 53.7157, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.437847
Model 0 best validation binary_cross_entropy = 0.324866 on epoch 208
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.267924
Ensemble test binary_cross_entropy = 0.267924
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 3,027,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 89.910461
Epoch 1
Validation binary_cross_entropy = 89.906044
Epoch 2
Validation binary_cross_entropy = 8.985806
Epoch 3
Validation binary_cross_entropy = 7.555851
Epoch 4
Loss = 7.3516e+01, PNorm = 60.3420, GNorm = 1368.8303, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 25.865435
Epoch 5
Validation binary_cross_entropy = 43.179432
Epoch 6
Validation binary_cross_entropy = 25.191662
Epoch 7
Validation binary_cross_entropy = 6.719181
Epoch 8
Validation binary_cross_entropy = 9.824004
Epoch 9
Loss = 4.9370e+01, PNorm = 60.4643, GNorm = 833.9003, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 66.633247
Epoch 10
Validation binary_cross_entropy = 76.663185
Epoch 11
Validation binary_cross_entropy = 20.508375
Epoch 12
Validation binary_cross_entropy = 6.991170
Epoch 13
Validation binary_cross_entropy = 5.583305
Epoch 14
Loss = 7.0871e+01, PNorm = 60.6060, GNorm = 1069.6421, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 32.582394
Epoch 15
Validation binary_cross_entropy = 66.948540
Epoch 16
Validation binary_cross_entropy = 28.741665
Epoch 17
Validation binary_cross_entropy = 4.902768
Epoch 18
Validation binary_cross_entropy = 5.323285
Epoch 19
Loss = 4.8106e+01, PNorm = 60.7309, GNorm = 626.8515, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 4.388348
Epoch 20
Validation binary_cross_entropy = 14.984299
Epoch 21
Validation binary_cross_entropy = 18.759007
Epoch 22
Validation binary_cross_entropy = 3.367170
Epoch 23
Validation binary_cross_entropy = 4.588591
Epoch 24
Loss = 3.5372e+01, PNorm = 60.8202, GNorm = 745.4927, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 4.312597
Epoch 25
Validation binary_cross_entropy = 3.275398
Epoch 26
Validation binary_cross_entropy = 18.177393
Epoch 27
Validation binary_cross_entropy = 6.407331
Epoch 28
Validation binary_cross_entropy = 3.829937
Epoch 29
Loss = 2.9289e+01, PNorm = 60.8899, GNorm = 772.7119, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 4.279077
Epoch 30
Validation binary_cross_entropy = 3.660348
Epoch 31
Validation binary_cross_entropy = 4.722032
Epoch 32
Validation binary_cross_entropy = 8.229925
Epoch 33
Validation binary_cross_entropy = 3.073865
Epoch 34
Loss = 2.2567e+01, PNorm = 60.9754, GNorm = 634.2618, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 3.620777
Epoch 35
Validation binary_cross_entropy = 3.411585
Epoch 36
Validation binary_cross_entropy = 2.366030
Epoch 37
Validation binary_cross_entropy = 11.757933
Epoch 38
Validation binary_cross_entropy = 5.681637
Epoch 39
Loss = 2.1779e+01, PNorm = 61.1154, GNorm = 611.1125, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.134843
Epoch 40
Validation binary_cross_entropy = 2.994765
Epoch 41
Validation binary_cross_entropy = 5.455728
Epoch 42
Validation binary_cross_entropy = 17.646397
Epoch 43
Validation binary_cross_entropy = 27.410091
Epoch 44
Loss = 2.2394e+01, PNorm = 61.2559, GNorm = 338.9661, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 8.465414
Epoch 45
Validation binary_cross_entropy = 5.172031
Epoch 46
Validation binary_cross_entropy = 6.728887
Epoch 47
Validation binary_cross_entropy = 15.348840
Epoch 48
Validation binary_cross_entropy = 17.596546
Epoch 49
Loss = 1.8505e+01, PNorm = 61.3776, GNorm = 477.7543, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 5.858129
Epoch 50
Validation binary_cross_entropy = 2.902944
Epoch 51
Validation binary_cross_entropy = 3.102235
Epoch 52
Validation binary_cross_entropy = 2.671827
Epoch 53
Validation binary_cross_entropy = 5.412390
Epoch 54
Loss = 1.5855e+01, PNorm = 61.4549, GNorm = 420.0301, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 10.465590
Epoch 55
Validation binary_cross_entropy = 4.343233
Epoch 56
Validation binary_cross_entropy = 3.255626
Epoch 57
Validation binary_cross_entropy = 2.929021
Epoch 58
Validation binary_cross_entropy = 2.826993
Epoch 59
Loss = 1.0962e+01, PNorm = 61.5107, GNorm = 334.2897, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 4.319207
Epoch 60
Validation binary_cross_entropy = 5.930008
Epoch 61
Validation binary_cross_entropy = 4.492769
Epoch 62
Validation binary_cross_entropy = 4.028879
Epoch 63
Validation binary_cross_entropy = 3.613613
Epoch 64
Loss = 1.4894e+01, PNorm = 61.5483, GNorm = 233.2004, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.903616
Epoch 65
Validation binary_cross_entropy = 2.277334
Epoch 66
Validation binary_cross_entropy = 2.324734
Epoch 67
Validation binary_cross_entropy = 2.272698
Epoch 68
Validation binary_cross_entropy = 2.484530
Epoch 69
Loss = 8.6827e+00, PNorm = 61.5796, GNorm = 331.0438, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.388218
Epoch 70
Validation binary_cross_entropy = 3.608728
Epoch 71
Validation binary_cross_entropy = 3.599778
Epoch 72
Validation binary_cross_entropy = 3.684583
Epoch 73
Validation binary_cross_entropy = 3.530841
Epoch 74
Loss = 1.5141e+01, PNorm = 61.6103, GNorm = 406.1085, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 6.888691
Epoch 75
Validation binary_cross_entropy = 7.294132
Epoch 76
Validation binary_cross_entropy = 3.880769
Epoch 77
Validation binary_cross_entropy = 3.000448
Epoch 78
Validation binary_cross_entropy = 4.337864
Epoch 79
Loss = 1.0461e+01, PNorm = 61.6515, GNorm = 508.2436, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.161847
Epoch 80
Validation binary_cross_entropy = 1.132849
Epoch 81
Validation binary_cross_entropy = 1.078860
Epoch 82
Validation binary_cross_entropy = 3.351045
Epoch 83
Validation binary_cross_entropy = 6.152346
Epoch 84
Loss = 8.6636e+00, PNorm = 61.6555, GNorm = 286.0687, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 5.940736
Epoch 85
Validation binary_cross_entropy = 3.287528
Epoch 86
Validation binary_cross_entropy = 2.261520
Epoch 87
Validation binary_cross_entropy = 1.888472
Epoch 88
Validation binary_cross_entropy = 1.196952
Epoch 89
Loss = 7.1364e+00, PNorm = 61.6736, GNorm = 208.8523, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.163058
Epoch 90
Validation binary_cross_entropy = 1.132504
Epoch 91
Validation binary_cross_entropy = 1.097839
Epoch 92
Validation binary_cross_entropy = 1.035471
Epoch 93
Validation binary_cross_entropy = 1.011723
Epoch 94
Loss = 8.3959e+00, PNorm = 61.6941, GNorm = 283.2651, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.996914
Epoch 95
Validation binary_cross_entropy = 1.015636
Epoch 96
Validation binary_cross_entropy = 1.039608
Epoch 97
Validation binary_cross_entropy = 1.026803
Epoch 98
Validation binary_cross_entropy = 0.956515
Epoch 99
Loss = 6.7016e+00, PNorm = 61.7132, GNorm = 259.6034, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.934391
Epoch 100
Validation binary_cross_entropy = 0.935437
Epoch 101
Validation binary_cross_entropy = 0.978062
Epoch 102
Validation binary_cross_entropy = 0.992433
Epoch 103
Validation binary_cross_entropy = 0.984157
Epoch 104
Loss = 5.9794e+00, PNorm = 61.7342, GNorm = 262.8595, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.988912
Epoch 105
Validation binary_cross_entropy = 0.959695
Epoch 106
Validation binary_cross_entropy = 0.951839
Epoch 107
Validation binary_cross_entropy = 0.934539
Epoch 108
Validation binary_cross_entropy = 0.837514
Epoch 109
Loss = 6.5075e+00, PNorm = 61.7525, GNorm = 415.2610, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.821684
Epoch 110
Validation binary_cross_entropy = 0.981084
Epoch 111
Validation binary_cross_entropy = 1.107904
Epoch 112
Validation binary_cross_entropy = 1.133204
Epoch 113
Validation binary_cross_entropy = 0.919039
Epoch 114
Loss = 3.1669e+00, PNorm = 61.7841, GNorm = 279.3902, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.887515
Epoch 115
Validation binary_cross_entropy = 0.872754
Epoch 116
Validation binary_cross_entropy = 0.855379
Epoch 117
Validation binary_cross_entropy = 0.825229
Epoch 118
Validation binary_cross_entropy = 0.779599
Epoch 119
Loss = 2.8818e+00, PNorm = 61.8255, GNorm = 129.0223, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.700487
Epoch 120
Validation binary_cross_entropy = 0.704900
Epoch 121
Validation binary_cross_entropy = 0.742735
Epoch 122
Validation binary_cross_entropy = 0.621495
Epoch 123
Validation binary_cross_entropy = 0.608965
Epoch 124
Loss = 3.3333e+00, PNorm = 61.8531, GNorm = 141.2353, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.653540
Epoch 125
Validation binary_cross_entropy = 0.861511
Epoch 126
Validation binary_cross_entropy = 0.869820
Epoch 127
Validation binary_cross_entropy = 0.628911
Epoch 128
Validation binary_cross_entropy = 0.540123
Epoch 129
Loss = 3.0802e+00, PNorm = 61.8548, GNorm = 127.6421, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.519503
Epoch 130
Validation binary_cross_entropy = 0.524751
Epoch 131
Validation binary_cross_entropy = 0.552985
Epoch 132
Validation binary_cross_entropy = 0.581351
Epoch 133
Validation binary_cross_entropy = 0.603388
Epoch 134
Loss = 3.4040e+00, PNorm = 61.8700, GNorm = 170.0134, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.686559
Epoch 135
Validation binary_cross_entropy = 0.631632
Epoch 136
Validation binary_cross_entropy = 0.625063
Epoch 137
Validation binary_cross_entropy = 0.604258
Epoch 138
Validation binary_cross_entropy = 0.535322
Epoch 139
Loss = 2.6516e+00, PNorm = 61.8952, GNorm = 173.9612, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.514482
Epoch 140
Validation binary_cross_entropy = 0.588058
Epoch 141
Validation binary_cross_entropy = 0.891348
Epoch 142
Validation binary_cross_entropy = 1.074070
Epoch 143
Validation binary_cross_entropy = 1.080155
Epoch 144
Loss = 4.4537e+00, PNorm = 61.9199, GNorm = 166.6550, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.043409
Epoch 145
Validation binary_cross_entropy = 0.667756
Epoch 146
Validation binary_cross_entropy = 0.465437
Epoch 147
Validation binary_cross_entropy = 0.408341
Epoch 148
Validation binary_cross_entropy = 0.399348
Epoch 149
Loss = 2.8973e+00, PNorm = 61.9559, GNorm = 120.2069, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.439869
Epoch 150
Validation binary_cross_entropy = 0.531328
Epoch 151
Validation binary_cross_entropy = 0.707669
Epoch 152
Validation binary_cross_entropy = 0.801056
Epoch 153
Validation binary_cross_entropy = 0.807550
Epoch 154
Loss = 3.1960e+00, PNorm = 61.9643, GNorm = 205.6622, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.833053
Epoch 155
Validation binary_cross_entropy = 0.940170
Epoch 156
Validation binary_cross_entropy = 0.898746
Epoch 157
Validation binary_cross_entropy = 0.830397
Epoch 158
Validation binary_cross_entropy = 0.672536
Epoch 159
Loss = 1.9125e+00, PNorm = 61.9726, GNorm = 120.2923, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.497442
Epoch 160
Validation binary_cross_entropy = 0.459701
Epoch 161
Validation binary_cross_entropy = 0.439704
Epoch 162
Validation binary_cross_entropy = 0.434254
Epoch 163
Validation binary_cross_entropy = 0.471649
Epoch 164
Loss = 3.4153e+00, PNorm = 61.9886, GNorm = 242.1975, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.462557
Epoch 165
Validation binary_cross_entropy = 0.529305
Epoch 166
Validation binary_cross_entropy = 0.632244
Epoch 167
Validation binary_cross_entropy = 0.689439
Epoch 168
Validation binary_cross_entropy = 0.653409
Epoch 169
Loss = 2.7217e+00, PNorm = 62.0178, GNorm = 136.7173, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.542279
Epoch 170
Validation binary_cross_entropy = 0.454943
Epoch 171
Validation binary_cross_entropy = 0.409530
Epoch 172
Validation binary_cross_entropy = 0.426078
Epoch 173
Validation binary_cross_entropy = 0.470952
Epoch 174
Loss = 1.5106e+00, PNorm = 62.0485, GNorm = 96.7838, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.514335
Epoch 175
Validation binary_cross_entropy = 0.707369
Epoch 176
Validation binary_cross_entropy = 0.891665
Epoch 177
Validation binary_cross_entropy = 1.038662
Epoch 178
Validation binary_cross_entropy = 1.121794
Epoch 179
Loss = 3.0012e+00, PNorm = 62.0478, GNorm = 130.0923, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.163861
Epoch 180
Validation binary_cross_entropy = 1.200949
Epoch 181
Validation binary_cross_entropy = 1.117331
Epoch 182
Validation binary_cross_entropy = 0.947459
Epoch 183
Validation binary_cross_entropy = 0.757118
Epoch 184
Loss = 2.1487e+00, PNorm = 62.0532, GNorm = 105.2239, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.635895
Epoch 185
Validation binary_cross_entropy = 0.559088
Epoch 186
Validation binary_cross_entropy = 0.558659
Epoch 187
Validation binary_cross_entropy = 0.578908
Epoch 188
Validation binary_cross_entropy = 0.612428
Epoch 189
Loss = 1.0563e+00, PNorm = 62.0621, GNorm = 48.7135, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.643947
Epoch 190
Validation binary_cross_entropy = 0.674624
Epoch 191
Validation binary_cross_entropy = 0.713166
Epoch 192
Validation binary_cross_entropy = 0.760063
Epoch 193
Validation binary_cross_entropy = 0.771873
Epoch 194
Loss = 2.1269e+00, PNorm = 62.0606, GNorm = 61.7994, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.774852
Epoch 195
Validation binary_cross_entropy = 0.713789
Epoch 196
Validation binary_cross_entropy = 0.655657
Epoch 197
Validation binary_cross_entropy = 0.581818
Epoch 198
Validation binary_cross_entropy = 0.553672
Epoch 199
Loss = 1.7482e+00, PNorm = 62.0678, GNorm = 137.7195, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.538371
Epoch 200
Validation binary_cross_entropy = 0.525653
Epoch 201
Validation binary_cross_entropy = 0.486240
Epoch 202
Validation binary_cross_entropy = 0.453116
Epoch 203
Validation binary_cross_entropy = 0.433722
Epoch 204
Loss = 2.5337e+00, PNorm = 62.0853, GNorm = 152.8093, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.430481
Epoch 205
Validation binary_cross_entropy = 0.428333
Epoch 206
Validation binary_cross_entropy = 0.463231
Epoch 207
Validation binary_cross_entropy = 0.514197
Epoch 208
Validation binary_cross_entropy = 0.588428
Epoch 209
Loss = 2.2785e+00, PNorm = 62.1120, GNorm = 66.3864, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.679164
Epoch 210
Validation binary_cross_entropy = 0.761554
Epoch 211
Validation binary_cross_entropy = 0.785580
Epoch 212
Validation binary_cross_entropy = 0.781028
Epoch 213
Validation binary_cross_entropy = 0.782073
Epoch 214
Loss = 1.8404e+00, PNorm = 62.1258, GNorm = 62.3517, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.756144
Epoch 215
Validation binary_cross_entropy = 0.732452
Epoch 216
Validation binary_cross_entropy = 0.712425
Epoch 217
Validation binary_cross_entropy = 0.675602
Epoch 218
Validation binary_cross_entropy = 0.619417
Epoch 219
Loss = 1.4409e+00, PNorm = 62.1389, GNorm = 105.6970, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.586381
Epoch 220
Validation binary_cross_entropy = 0.562677
Epoch 221
Validation binary_cross_entropy = 0.531784
Epoch 222
Validation binary_cross_entropy = 0.521121
Epoch 223
Validation binary_cross_entropy = 0.524460
Epoch 224
Loss = 2.1540e+00, PNorm = 62.1495, GNorm = 111.5908, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.546632
Epoch 225
Validation binary_cross_entropy = 0.561916
Epoch 226
Validation binary_cross_entropy = 0.572022
Epoch 227
Validation binary_cross_entropy = 0.569331
Epoch 228
Validation binary_cross_entropy = 0.552499
Epoch 229
Loss = 1.3346e+00, PNorm = 62.1799, GNorm = 64.4248, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.542015
Epoch 230
Validation binary_cross_entropy = 0.535282
Epoch 231
Validation binary_cross_entropy = 0.504987
Epoch 232
Validation binary_cross_entropy = 0.464658
Epoch 233
Validation binary_cross_entropy = 0.455637
Epoch 234
Loss = 1.5495e+00, PNorm = 62.1904, GNorm = 104.5805, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.441089
Epoch 235
Validation binary_cross_entropy = 0.434017
Epoch 236
Validation binary_cross_entropy = 0.436087
Epoch 237
Validation binary_cross_entropy = 0.446252
Epoch 238
Validation binary_cross_entropy = 0.463798
Epoch 239
Loss = 1.6081e+00, PNorm = 62.1862, GNorm = 120.0917, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.475013
Epoch 240
Validation binary_cross_entropy = 0.477815
Epoch 241
Validation binary_cross_entropy = 0.455931
Epoch 242
Validation binary_cross_entropy = 0.448658
Epoch 243
Validation binary_cross_entropy = 0.448265
Epoch 244
Loss = 7.3077e-01, PNorm = 62.2033, GNorm = 73.3390, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.457590
Epoch 245
Validation binary_cross_entropy = 0.472821
Epoch 246
Validation binary_cross_entropy = 0.491869
Epoch 247
Validation binary_cross_entropy = 0.502608
Epoch 248
Validation binary_cross_entropy = 0.516988
Epoch 249
Loss = 9.1371e-01, PNorm = 62.2202, GNorm = 40.8716, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.531341
Epoch 250
Validation binary_cross_entropy = 0.540011
Epoch 251
Validation binary_cross_entropy = 0.550090
Epoch 252
Validation binary_cross_entropy = 0.568235
Epoch 253
Validation binary_cross_entropy = 0.560171
Epoch 254
Loss = 1.3613e+00, PNorm = 62.2264, GNorm = 72.3448, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.549159
Epoch 255
Validation binary_cross_entropy = 0.532196
Epoch 256
Validation binary_cross_entropy = 0.501210
Epoch 257
Validation binary_cross_entropy = 0.477880
Epoch 258
Validation binary_cross_entropy = 0.456122
Epoch 259
Loss = 6.5910e-01, PNorm = 62.2247, GNorm = 40.8983, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.444851
Epoch 260
Validation binary_cross_entropy = 0.440034
Epoch 261
Validation binary_cross_entropy = 0.435619
Epoch 262
Validation binary_cross_entropy = 0.449608
Epoch 263
Validation binary_cross_entropy = 0.467919
Epoch 264
Loss = 6.7521e-01, PNorm = 62.2214, GNorm = 46.7501, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.478957
Epoch 265
Validation binary_cross_entropy = 0.485943
Epoch 266
Validation binary_cross_entropy = 0.453287
Epoch 267
Validation binary_cross_entropy = 0.402753
Epoch 268
Validation binary_cross_entropy = 0.398796
Epoch 269
Loss = 9.9137e-01, PNorm = 62.2322, GNorm = 70.4062, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.384132
Epoch 270
Validation binary_cross_entropy = 0.407211
Epoch 271
Validation binary_cross_entropy = 0.435537
Epoch 272
Validation binary_cross_entropy = 0.461838
Epoch 273
Validation binary_cross_entropy = 0.468174
Epoch 274
Loss = 1.5914e+00, PNorm = 62.2417, GNorm = 136.6253, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.479515
Epoch 275
Validation binary_cross_entropy = 0.484869
Epoch 276
Validation binary_cross_entropy = 0.492734
Epoch 277
Validation binary_cross_entropy = 0.498969
Epoch 278
Validation binary_cross_entropy = 0.518482
Epoch 279
Loss = 7.4228e-01, PNorm = 62.2452, GNorm = 51.9418, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.527419
Epoch 280
Validation binary_cross_entropy = 0.552956
Epoch 281
Validation binary_cross_entropy = 0.564057
Epoch 282
Validation binary_cross_entropy = 0.561215
Epoch 283
Validation binary_cross_entropy = 0.563074
Epoch 284
Loss = 7.1440e-01, PNorm = 62.2503, GNorm = 52.6590, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.552685
Epoch 285
Validation binary_cross_entropy = 0.512240
Epoch 286
Validation binary_cross_entropy = 0.480954
Epoch 287
Validation binary_cross_entropy = 0.457674
Epoch 288
Validation binary_cross_entropy = 0.446388
Epoch 289
Loss = 1.1229e+00, PNorm = 62.2566, GNorm = 62.7319, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.441236
Epoch 290
Validation binary_cross_entropy = 0.447655
Epoch 291
Validation binary_cross_entropy = 0.476329
Epoch 292
Validation binary_cross_entropy = 0.497020
Epoch 293
Validation binary_cross_entropy = 0.517306
Epoch 294
Loss = 6.8402e-01, PNorm = 62.2667, GNorm = 53.9535, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.524953
Epoch 295
Validation binary_cross_entropy = 0.510530
Epoch 296
Validation binary_cross_entropy = 0.482167
Epoch 297
Validation binary_cross_entropy = 0.452849
Epoch 298
Validation binary_cross_entropy = 0.429520
Epoch 299
Loss = 9.2360e-01, PNorm = 62.2875, GNorm = 55.1276, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.398794
Model 0 best validation binary_cross_entropy = 0.384132 on epoch 269
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.397356
Ensemble test binary_cross_entropy = 0.397356
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.445539
	Seed 1 ==> test binary_cross_entropy = 0.403390
	Seed 2 ==> test binary_cross_entropy = 0.396205
	Seed 3 ==> test binary_cross_entropy = 0.316820
	Seed 4 ==> test binary_cross_entropy = 0.279857
	Seed 5 ==> test binary_cross_entropy = 0.303652
	Seed 6 ==> test binary_cross_entropy = 0.393767
	Seed 7 ==> test binary_cross_entropy = 0.369676
	Seed 8 ==> test binary_cross_entropy = 0.267924
	Seed 9 ==> test binary_cross_entropy = 0.397356
Overall test binary_cross_entropy = 0.357419 +/- 0.057458
Elapsed time = 0:08:09
